How do I pass LSST_LIBRARY_PATH through scons through assertExecutable to a bash script?

Support
,
#1

How do I pass LSST_LIBRARY_PATH or DYLD_LIBRARY_PATH through scons through assertExecutable to a bash script?

I’d like to do the below (as part of lsst_ci). This works in a py.test environment, but DYLD_LIBRARY_PATH nor LSST_LIBRARY_PATH are passed through to the underlying bash script.

I know that this is a SIP issue. I’m asking for a solution.

executable_dir = os.path.join(
    lsst.utils.getPackageDir("VALIDATE_DRP"),
    "examples")


class ExampleObsTestCase(lsst.utils.tests.ExecutablesTestCase):
    """Test an example obs_ processing run."""
    def testObsCfhtQuick(self):
        """Test obs_cfht"""
        self.assertExecutable("runCfhtQuickTest.sh",
                              root_dir=executable_dir,
                              msg="CFHT Quick Test failed")

    def testObsDecamQuick(self):
        """Test obs_decam"""
        self.assertExecutable("runDecamQuickTest.sh",
                              root_dir=executable_dir,
                              msg="DECam Quick Test failed")


if __name__ == "__main__":
    lsst.utils.tests.init()
    unittest.main()
#2

Explicitly adding LSST_LIBRARY_PATH in sconsUtils.tests in Control.run works.

            libpathstr = utils.libraryLoaderEnvironment()
            libpathstr += ' {}="{}"'.format('LSST_LIBRARY_PATH', os.getenv('LSST_LIBRARY_PATH'))

This seems naively to indicate that both LSST_LIBRARY_PATH and DYLD_LIBRARY_PATH should be passed to ensure success.

#3

The bash script being called in the test explicitly copies LSST_LIBRARY_PATH to DYLD_LIBRARY_PATH.

#4

In Galsim, we explicitly prepend DYLD_LIBRARY_PATH (et al) before the executable name when we need to run executables from within SCons. cf. https://github.com/GalSim-developers/GalSim/blob/releases/1.4/SConstruct#L751 for the helper function that does the work.

#5

That’s what we do to. The problem here is that the script is a bash script so when it runs the library path gets stripped straight away.

#6

Ah. I see. I guess I didn’t fully appreciate the insidiousness that is SIP. I haven’t upgraded past Yosemite myself yet, so it’s all hearsay to me… :slight_smile:

#7

Exactly. The python scripts need DYLD_LIBRARY_PATH and the bash scripts need LSST_LIBRARY_PATH so they can copy the environment to DYLD_LIBRARY_PATH.

#8

sconsUtils appears to instead make an XOR choice.

def libraryLoaderEnvironment():
    libpathstr = ""
    # If we have an OS X with System Integrity Protection enabled or similar we need
    # to pass through DYLD_LIBRARY_PATH to the external command.
    pass_through_var = libraryPathPassThrough()
    if pass_through_var is not None:
        for varname in (pass_through_var, "LSST_LIBRARY_PATH"):
            if varname in os.environ:
                libpathstr = '{}="{}"'.format(pass_through_var, os.environ[varname])
                break
return libpathstr

Should this be changed to always providing both on Darwin?

#9

https://jira.lsstcorp.org/browse/DM-9794
"Pass both LSST_LIBRARY_PATH and DYLD_LIBRARY_PATH in scons on Mac OS"

#10

https://jira.lsstcorp.org/browse/DM-9794

now implemented and merged to master.