We could use ASAN builds

Sure enough, the existing cbuild_config.py has a way to build the bindings with address sanitizer. But, it only works for OS X. Also, on OS X asan doesn't detect leaks, you have to use leak sanitizer but on linux I believe it does both. Ideally we'd check for leaks as well.

Python (at least on OS X) uses shim executables which spawn python in the first place. You need to force the loading of the dynamic asan library (LD_PRELOAD/DYLD_INSERT_LIBRARIES, which seems not to work well across the shims. This makes using in OS X a bit of a PITA - you have to find the actual python executable (buried 2 shims and one symlink deep on my machine), and execute that. Doing this on jenkins seems.... unlikely to work reliably.

Lets do this: Add a bit so we can use linux (maybe windows? maybe not). Then I believe we can automate an asan build (say, of all the tests) perhaps. Linux AFAIK doesn't have the shim difficulty. In any case, would be good to be able to run it on both, as different platforms may have different issues. Also, asan on linux includes the leak detection, which is really important to have too.

NOTE: This isn't critical to do right away. We can do by hand for the current release. Also - checkout this for a discussion of the shims and so on.