Pin numpy version? And build/packaging questions

[jnj16180340]

Re: builds and packaging:

Last night, I built a docker image including javabridge and it ran fine. This morning I rebuilt it and importing javabridge failed with the dreaded ValueError: numpy.ufunc has the wrong size, try recompiling.

Between last night and this morning:

• No change in container definition
• No change in versions of installed system + python packages
• But SHA256 of javabridge installed files did change
• numpy released 1.16.0rc1, not installed in my container because it's a prerelease

Either one, or both, of these is sufficient to fix the broken import:

• Manually install numpy 1.16.0rc1 in the container
• Install javabridge from source w/ pip install --no-binary javabridge -r requirements.txt

So, between last night and this morning the javabridge installed in my container went from being compiled against numpy 1.15 to compiled against 1.16. I'm wondering how this happened so it doesn't happen again.

My hypothesis about what happened:

• numpy is unversioned in javabridge requiments
• javabridge rebuilt on numpy 1.16rc1
• pip now installs numpy 15.x and javabridge(numpy 1.16)

In this case, pinning numpy version could help.

But, this doesn't make sense because pypi.org shows javabridge was last updated in September! And I just don't believe anyone would automate deployment based on dependency prereleases without at least incrementing the version.

I also suspect the problem involves my misunderstanding of pypi source distributions, binary distributions, pip, etc. I'd totally appreciate it if someone has insight.

Also generally, can anyone shed some light on how javabridge is built and published?

[LeeKamentsky]

Answering the question about Javabridge builds and publishing - it pretty much has to be installed from source, unless you use something like Anaconda to keep track of version compatibility. It needs to be built against both Numpy and the JVM which both have their versions, so it would be difficult to have binaries for every cross-combination.
I bump the version of Javabridge pretty much whenever someone has a compelling need for a new version on PyPi, most notably, if CellProfiler needs it. But you can pretty much ask and I'll cut a new version and publish it within a few days. Javabridge is at this point stable, except for the complexities of building it and I haven't added features in a while.

Regarding Javabridge changing out from under you, I can't think of any circumstances other than "pip install" that would do that. The best I can think of is that Python was finding Javabridge in two different places depending on how Python was run.