OSX Install Notes

OS X Specific Notes

Apple’s implementation of BLAS does not support using BLAS calls on both sides of a fork. The upshot of this is that, when using NumPy functions that rely on BLAS calls within a forked process (such as ones created when you push a job into a multiprocessing pool) the fork might never actually fully exit. Which means you end up with orphaned processes until the process that was originally forked exits. Since under the hood Dedupe relies upon NumPy calls within a multiprocessing pool, this can be an issue if you are planning on running something like a daemon process that then forks off processes that run Dedupe.

One way to get around this is to compile NumPy against a different implementation of BLAS such as OpenBLAS. Here’s how you might go about that:

Clone and build OpenBLAS source with USE_OPENMP=0 flag

$ git clone https://github.com/xianyi/OpenBLAS.git
$ cd OpenBLAS
$ make USE_OPENMP=0
$ mkdir /usr/local/opt/openblas # Change this to suit your needs
$ make PREFIX=/usr/local/opt/openblas install # Make sure this matches the path above

Clone and build NumPy

Make sure it knows where you just built OpenBLAS. This involves editing the site.cfg file within the NumPy source (see http://stackoverflow.com/a/14391693/1907889 for details). The paths that you’ll enter in there are relative to the ones used in step one above.

The Homebrew Science formulae also offer an OpenBLAS formula but as of this writing it was still referencing the current release of OpenBLAS (0.2.8) which does not include a fix for a bug which is the whole reason this is necessary in the first place. Once that fix is rolled into a release and the Homebrew formula is updated, this will be a better approach to getting this setup.

Alternatively, compile NumPy with no BLAS support

According to their documentation, NumPy does not actually require Linear Algebra libraries (such as OpenBLAS) to be installed. But, since by default it is configured to look for and use these libraries (and since OS X technically ships with one), you’ll need to tell NumPy to compile without looking them. To accomplish this, you can use environmental variables like so:

$ export BLAS=None
$ export LAPACK=None
$ export ATLAS=None
$ cd /path/to/numpy/source
$ python setup.py build
$ python setup.py install