EUPS binary "tarballs" for OSX are back; conda env update; osx-10.11 builds dropped

Data Management DM Notifications
, ,
#1

There are several recent changes that EUPS binary “tarball” users should be aware of:

  • As of this morning, nightly/weekly binary OSX “tarballs” should again be produced regularly. For those unaware, osx binary builds were disabled on August 2nd due to build failures that we suspect were being caused by a middleware box attempting to filter https connections. d_2018_08_28 and d_2018_08_29 have been published. (DM-15558)

  • Starting with d_2018_08_28, the lsstsw / newinstall.sh conda environment has been updated to fcd27eb: linux / osx (DM-14011 & DM-14624)

    • Of particular note, the python interpreter is now 3.6.6 and scikit-learn has been added in addition to a general upgrade of all conda package versions.
    • As a result of the env update, the URL for the current EUPS_PKGROOT for binary “tarballs” has changed for all platforms.

  • Prior to yesterday, the jenkins stack-os-matrix job would run on the osx build on either 10.11 (elcap) or 10.12 (sierra) and the binary tarball build only on 10.11. Due to the build of astshim failing on 10.11 (this was only randomly see with stack-os-matrix as some of builds were landing on 10.12) after the conda env upgrade, building on 10.11 has been dropped and all OSX builds will presently run on 10.12. As a different version of xcode is installed in our 10.12 build env, the “compiler string” for osx binary tarballs has also changed to clang-802.0.42. Eg., https://eups.lsst.codes/stack/osx/10.9/clang-802.0.42/

newinstall.sh users

There is some discussion of tarball configuration on the install/newinstall page.

An existing “newinstall” may be updated by rerunning newinstall.sh from the root directory. Note that this will start a new EUPS_PATH. Eg.

$ docker run -ti lsstsqre/centos:7-stack-lsst_distrib-w_2018_34
[lsst@7f25d69f6bd7 stack]$ curl -sSL https://raw.githubusercontent.com/lsst/lsst/master/scripts/newinstall.sh | bash -s -- -cbtS

LSST Software Stack Builder
=======================================================================

....
[lsst@7f25d69f6bd7 stack]$ ls -la stack/
total 28
drwxrwxr-x. 1 lsst lsst 4096 Aug 29 21:06 .
drwxr-xr-x. 1 lsst lsst 4096 Aug 12 17:32 ..
lrwxrwxrwx. 1 lsst lsst   24 Aug 29 21:06 current -> miniconda3-4.5.4-fcd27eb
drwxrwxr-x. 1 lsst lsst 4096 Aug 25 14:09 miniconda3-4.5.4-10a4fa6
drwxrwxr-x. 4 lsst lsst 4096 Aug 29 21:06 miniconda3-4.5.4-fcd27eb
Changes to baseline software versions
#2

Given that we force MACOSX_DEPLOYMENT_TARGET (currently to an ancient 10.9) are we sure that the details of the clang compiler version are relevant? The idea is meant to be that our binaries are compatible with macOS 10.9 and later. It’s of course possible that we aren’t specifying MACOSX_DEPLOYMENT_TARGET consistently in the build and we should also consider changing it to 10.12 if we really aren’t worried about the binaries working on older macOS versions.

#3

Are we sure that it isn’t? clang seems to only claim partial c++ abi compatibility with gcc (http://libcxx.llvm.org/) and gcc's abi is essentially only guaranteed to be stable for patch releases. https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html

The OSX path tree follows the same as that of linux and the build compiler has certainly been significant there in the past.

#4

What are the new values?

#5

Looks like: