Failure to build v13_0 from source with user installed Anaconda

Support
#1

On RH6-64, with devtoolset-3 enabled, and my own vanilla Anaconda2 4.3.1 installed, I receive the following error after running eups distrib install -t v13_0 lsst_distrib --nolocks

  [ 37/93 ]  tmv 0.73+5                                                 done. 
...
  [ 44/93 ]  galsim 1.4.1.lsst3 ... 

***** error: from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.log:
TMV version is 0.73
Using TMV_LINK file: /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/share/tmv/tmv-link
     -L/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/lib -ltmv -lpthread -fopenmp
Checking for correct TMV linkage... (this may take a little while)
Checking for correct TMV linkage... yes
Checking if we can build against Python... 
Unable to build a python loadable module using the python executable:
/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/bin/python,
the library name libpython2.7.so,
and the libdir /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/lib.
Nor did any of the known variations on this work.
If these are not the correct library name or location, you can tell scons
the correct names to use with the flags EXTRA_LIB_PATH and/or EXTRA_LIBS.

Please fix the above error(s) and rerun scons.
Note: you may want to look through the file INSTALL.md for advice.
Also, if you are having trouble, please check the INSTALL FAQ at 
   https://github.com/GalSim-developers/GalSim/wiki/Installation%20FAQ

+ exit -4
eups distrib: Failed to build galsim-1.4.1.lsst3.eupspkg: Command:
    source /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/eups/bin/setups.sh; export EUPS_PATH=/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0; (/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.sh) >> /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.log 2>&1 4>/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.msg 
exited with code 252

As suggested I read the Install.md file and tested which python modules I had available - I am missing pyfits, future, and the optional starlink.Ast. This page just lists matplotlib and scipy as dependencies:
https://pipelines.lsst.io/install/newinstall.html#source-install-py-deps
I’ve now installed pyfits and future and restarted the build… it failed in the same way. To be fair, perhaps I should start fresh or at least clean out the galsim build products and try again.

If I allow newinstall to install Miniconda, the build is successful, so this is likely related to my use of a user installed Anaconda.

Poking around I see some discussion of TMV and galsim issues:
https://github.com/GalSim-developers/GalSim/wiki/Installation-FAQ#why-is-scons-unable-to-build-a-python-loadable-module-that-uses-tmv-using-anaconda

#2

The galsim dependencies are part of the build (future and pyfits) so they are not the problem. The fact that you did not have future installed on your python suggests to me that you are not using a miniconda recently installed by newinstall.sh and have an old one. I am wondering if your python is out of date and if you need to do conda update python as discussed in DM-7620 (which I thought was a Mac problem but it’s worth a try).

#3

I believe Anaconda2 4.3.1 is their most recent release but here is the result of running conda update python

bash-4.1$ conda update python 
Fetching package metadata ...........
Solving package specifications: .

# All requested packages already installed.
# packages in environment at /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1:
#
python                    2.7.13                        0

Sometimes I peek at the package list in https://github.com/lsst/lsstsw/blob/master/etc/conda2_packages-linux-64.txt and compare it to my own env. I already see some examples where I have a more recent version (astropy) than what might be required.

             Using externally provided astropy v1.3.

At this point I would typically shrug and create a conda env using the lsstsw package list and carry on by using that version to do this installation of v13_0 (of even more likely just let newinstall handle the miniconda installation for me). Is it wrong for me to assume that I really can’t use my own Anaconda installation? I note I don’t have nomkl… and hence my numpy, scipy etc are not nomkl either…

Here’s my conda env:

bash-4.1$ conda list
# packages in environment at /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1:
#
_license                  1.1                      py27_1  
alabaster                 0.7.9                    py27_0  
anaconda                  4.3.1               np111py27_0  
anaconda-client           1.6.0                    py27_0  
anaconda-navigator        1.5.0                    py27_0  
anaconda-project          0.4.1                    py27_0  
argcomplete               1.0.0                    py27_1  
astroid                   1.4.9                    py27_0  
astropy                   1.3                 np111py27_0  
babel                     2.3.4                    py27_0  
backports                 1.0                      py27_0  
backports_abc             0.5                      py27_0  
beautifulsoup4            4.5.3                    py27_0  
bitarray                  0.8.1                    py27_0  
blaze                     0.10.1                   py27_0  
bokeh                     0.12.4                   py27_0  
boto                      2.45.0                   py27_0  
bottleneck                1.2.0               np111py27_0  
cairo                     1.14.8                        0  
cdecimal                  2.3                      py27_2  
cffi                      1.9.1                    py27_0  
chardet                   2.3.0                    py27_0  
chest                     0.2.3                    py27_0  
click                     6.7                      py27_0  
cloudpickle               0.2.2                    py27_0  
clyent                    1.2.2                    py27_0  
colorama                  0.3.7                    py27_0  
conda                     4.3.14                   py27_0  
conda-env                 2.6.0                         0  
configobj                 5.0.6                    py27_0  
configparser              3.5.0                    py27_0  
contextlib2               0.5.4                    py27_0  
cryptography              1.7.1                    py27_0  
curl                      7.52.1                        0  
cycler                    0.10.0                   py27_0  
cython                    0.25.2                   py27_0  
cytoolz                   0.8.2                    py27_0  
dask                      0.13.0                   py27_0  
datashape                 0.5.4                    py27_0  
dbus                      1.10.10                       0  
decorator                 4.0.11                   py27_0  
dill                      0.2.5                    py27_0  
docutils                  0.13.1                   py27_0  
entrypoints               0.2.2                    py27_0  
enum34                    1.1.6                    py27_0  
et_xmlfile                1.0.1                    py27_0  
expat                     2.1.0                         0  
fastcache                 1.0.2                    py27_1  
flask                     0.12                     py27_0  
flask-cors                3.0.2                    py27_0  
fontconfig                2.12.1                        2  
freetype                  2.5.5                         2  
funcsigs                  1.0.2                    py27_0  
functools32               3.2.3.2                  py27_0  
future                    0.16.0                   py27_1  
futures                   3.0.5                    py27_0  
get_terminal_size         1.0.0                    py27_0  
gevent                    1.2.1                    py27_0  
glib                      2.50.2                        1  
greenlet                  0.4.11                   py27_0  
grin                      1.2.1                    py27_3  
gst-plugins-base          1.8.0                         0  
gstreamer                 1.8.0                         0  
h5py                      2.6.0               np111py27_2  
harfbuzz                  0.9.39                        2  
hdf5                      1.8.17                        1  
heapdict                  1.0.0                    py27_1  
icu                       54.1                          0  
idna                      2.2                      py27_0  
imagesize                 0.7.1                    py27_0  
ipaddress                 1.0.18                   py27_0  
ipykernel                 4.5.2                    py27_0  
ipython                   5.1.0                    py27_0  
ipython_genutils          0.1.0                    py27_0  
ipywidgets                5.2.2                    py27_1  
isort                     4.2.5                    py27_0  
itsdangerous              0.24                     py27_0  
jbig                      2.1                           0  
jdcal                     1.3                      py27_0  
jedi                      0.9.0                    py27_1  
jinja2                    2.9.4                    py27_0  
jpeg                      9b                            0  
jsonschema                2.5.1                    py27_0  
jupyter                   1.0.0                    py27_3  
jupyter_client            4.4.0                    py27_0  
jupyter_console           5.0.0                    py27_0  
jupyter_core              4.2.1                    py27_0  
lazy-object-proxy         1.2.2                    py27_0  
libffi                    3.2.1                         1  
libgcc                    4.8.5                         2  
libgfortran               3.0.0                         1  
libiconv                  1.14                          0  
libpng                    1.6.27                        0  
libsodium                 1.0.10                        0  
libtiff                   4.0.6                         3  
libxcb                    1.12                          1  
libxml2                   2.9.4                         0  
libxslt                   1.1.29                        0  
llvmlite                  0.15.0                   py27_0  
locket                    0.2.0                    py27_1  
lxml                      3.7.2                    py27_0  
markupsafe                0.23                     py27_2  
matplotlib                2.0.0               np111py27_0  
mistune                   0.7.3                    py27_0  
mkl                       2017.0.1                      0  
mkl-service               1.1.2                    py27_3  
mpmath                    0.19                     py27_1  
multipledispatch          0.4.9                    py27_0  
nbconvert                 4.2.0                    py27_0  
nbformat                  4.2.0                    py27_0  
networkx                  1.11                     py27_0  
nltk                      3.2.2                    py27_0  
nose                      1.3.7                    py27_1  
notebook                  4.3.1                    py27_0  
numba                     0.30.1              np111py27_0  
numexpr                   2.6.1               np111py27_2  
numpy                     1.11.3                   py27_0  
numpydoc                  0.6.0                    py27_0  
odo                       0.5.0                    py27_1  
openpyxl                  2.4.1                    py27_0  
openssl                   1.0.2k                        1  
pandas                    0.19.2              np111py27_1  
partd                     0.3.7                    py27_0  
path.py                   10.0                     py27_0  
pathlib2                  2.2.0                    py27_0  
patsy                     0.4.1                    py27_0  
pcre                      8.39                          1  
pep8                      1.7.0                    py27_0  
pexpect                   4.2.1                    py27_0  
pickleshare               0.7.4                    py27_0  
pillow                    4.0.0                    py27_0  
pip                       9.0.1                    py27_1  
pixman                    0.34.0                        0  
ply                       3.9                      py27_0  
prompt_toolkit            1.0.9                    py27_0  
psutil                    5.0.1                    py27_0  
ptyprocess                0.5.1                    py27_0  
py                        1.4.32                   py27_0  
pyasn1                    0.1.9                    py27_0  
pycairo                   1.10.0                   py27_0  
pycosat                   0.6.1                    py27_1  
pycparser                 2.17                     py27_0  
pycrypto                  2.6.1                    py27_4  
pycurl                    7.43.0                   py27_2  
pyfits                    3.4                       <pip>
pyflakes                  1.5.0                    py27_0  
pygments                  2.1.3                    py27_0  
pylint                    1.6.4                    py27_1  
pyopenssl                 16.2.0                   py27_0  
pyparsing                 2.1.4                    py27_0  
pyqt                      5.6.0                    py27_2  
pytables                  3.3.0               np111py27_0  
pytest                    3.0.5                    py27_0  
python                    2.7.13                        0  
python-dateutil           2.6.0                    py27_0  
pytz                      2016.10                  py27_0  
pyyaml                    3.12                     py27_0  
pyzmq                     16.0.2                   py27_0  
qt                        5.6.2                         3  
qtawesome                 0.4.3                    py27_0  
qtconsole                 4.2.1                    py27_1  
qtpy                      1.2.1                    py27_0  
readline                  6.2                           2  
redis                     3.2.0                         0  
redis-py                  2.10.5                   py27_0  
requests                  2.12.4                   py27_0  
rope                      0.9.4                    py27_1  
ruamel_yaml               0.11.14                  py27_1  
scandir                   1.4                      py27_0  
scikit-image              0.12.3              np111py27_1  
scikit-learn              0.18.1              np111py27_1  
scipy                     0.18.1              np111py27_1  
seaborn                   0.7.1                    py27_0  
setuptools                27.2.0                   py27_0  
simplegeneric             0.8.1                    py27_1  
singledispatch            3.4.0.3                  py27_0  
sip                       4.18                     py27_0  
six                       1.10.0                   py27_0  
snowballstemmer           1.2.1                    py27_0  
sockjs-tornado            1.0.3                    py27_0  
sphinx                    1.5.1                    py27_0  
spyder                    3.1.2                    py27_0  
sqlalchemy                1.1.5                    py27_0  
sqlite                    3.13.0                        0  
ssl_match_hostname        3.4.0.2                  py27_1  
statsmodels               0.6.1               np111py27_1  
subprocess32              3.2.7                    py27_0  
sympy                     1.0                      py27_0  
terminado                 0.6                      py27_0  
tk                        8.5.18                        0  
toolz                     0.8.2                    py27_0  
tornado                   4.4.2                    py27_0  
traitlets                 4.3.1                    py27_0  
unicodecsv                0.14.1                   py27_0  
wcwidth                   0.1.7                    py27_0  
werkzeug                  0.11.15                  py27_0  
wheel                     0.29.0                   py27_0  
widgetsnbextension        1.2.6                    py27_0  
wrapt                     1.10.8                   py27_0  
xlrd                      1.0.0                    py27_0  
xlsxwriter                0.9.6                    py27_0  
xlwt                      1.2.0                    py27_0  
xz                        5.2.2                         1  
yaml                      0.1.6                         0  
zeromq                    4.1.5                         0  
zlib                      1.2.8                         3
#4

Unfortunately galsim isn’t reporting the actual error message coming from the linker so it’s hard to work out what the problem is. Your anaconda versions are irrelevant for the link test. I assume the library path it is complaining about is the library path you are expecting it to be using.

It would be great if you could try a clean install with a miniconda installed from newinstall.sh just to see if this is a fundamental issue with your system or a weird issue with your pre-installed NFS-mounted anaconda.

@rmjarvis is there an easy way to convince GalSim to tell us what the link problem really is?

This will break meas_base (see DM-8146).

#5

As I think I noted at the end of my original post - the build is successful if I ignore my own Anaconda installation and allow newinstall to handle the miniconda installation.

1 Like
#6

Sorry. I missed that final sentence.

#7

Concerning nomkl, if I ever hope to use my own Anaconda installation, I should install nomkl and then set about obtaining the proper nomkl versions of numpy, scipy?

At this point - I fully expect I’ll retreat and just let newinstall just pull in miniconda… but there’s a part of me that would like to be able to use my own Anaconda python installation as long as I understand what it is that is necessary to do so.

#8

Yes. That’s unfortunately what is needed. Conda will work out what needs changing.

Agreed. It seems the only substantive differences are python 2.7.12 vs python 2.7.13 and conda 4.2.13 vs conda 4.3.14. One investigative route is to update the LSST-installed miniconda to use the newer python and newer conda and see if galsim fails to build with that configuration.

#9

@rmjarvis is there an easy way to convince GalSim to tell us what the link problem really is?

Look in the file gs_error.txt, which shows all the various compile and link attempts GalSim made and their outputs. We don’t report them all to the screen, since it typically tries a number of different possibilities, so it can be a little tricky figuring out which compilation attempt is the one that should have worked on your system. But all the attempts are in that file, along with a bunch of other useful information to help diagnose installation failures.

@heather999 If you want to send me that file, I can take a look and try to figure out what the problem is.

I will say off the cuff that the most common problem with our installation is boost. Especially in connection to Anaconda. cf. this FAQ entry.

#10

Correction. The file is called gs.error in v1.4.

#11

Thanks, @rmjarvis It’s a useful log :slight_smile: I’ve started to poke around, but I suspect you’ll see the problem a lot faster.
gs.error (344.9 KB)

#12

OK, Here is the bit that I think probably should have worked and didn’t:

.sconf_temp/conftest_36.cpp <-
  |
  |#include "Python.h"
  |
  |static PyObject* run(PyObject* self, PyObject* args)
  |{ return Py_BuildValue("i", 23); }
  |
  |static PyMethodDef Methods[] = {
  |    {"run",  run, METH_VARARGS, "return 23"},
  |    {NULL, NULL, 0, NULL}
  |};
  |
  |// cf. https://docs.python.org/3/howto/cporting.html#module-initialization-and-state
  |
  |#if PY_MAJOR_VERSION >= 3
  |
  |static struct PyModuleDef moduledef = {
  |    PyModuleDef_HEAD_INIT,
  |    "check_python",
  |    NULL,
  |    -1,
  |    Methods,
  |    NULL,
  |    NULL,
  |    NULL,
  |    NULL
  |};
  |
  |PyMODINIT_FUNC PyInit_check_python(void)
  |
  |#else
  |
  |PyMODINIT_FUNC initcheck_python(void)
  |
  |#endif
  |{
  |#if PY_MAJOR_VERSION >= 3
  |    return PyModule_Create(&moduledef);
  |#else
  |    Py_InitModule("check_python", Methods);
  |#endif
  |}
  |
g++ -o .sconf_temp/conftest_36.os -c -O2 -fno-strict-aliasing -fPIC -Iinclude -I/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/boost/1.60.lsst1+1/include -I/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/include -I/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/fftw/3.3.4.lsst2/include -I/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/include -I/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/include/python2.7 .sconf_temp/conftest_36.cpp
g++ -o .sconf_temp/conftest_36_mod/check_python.so -fopenmp -shared .sconf_temp/conftest_36.os -Llib -L/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/boost/1.60.lsst1+1/lib -L/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/lib -L/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/fftw/3.3.4.lsst2/lib -L/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/lib -lpython2.7 -ltmv_symband -lfftw3 -lpthread -ltmv -lpthread
.sconf_temp/conftest_37 <-
  |import sys
  |sys.path.append('.sconf_temp/conftest_36_mod')
  |import check_python
  |print(check_python.run())
bash -c '/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/bin/python' < .sconf_temp/conftest_37 > .sconf_temp/conftest_37.out 2>&1

Unfortunately, I’m not sure I understand the error. When it tries to print out what happens when this command is run, it says:

Output of the command '/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/bin/python < .sconf_temp/conftest_37' is:
bash: /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/bin/python < .sconf_temp/conftest_37: No such file or directory

So is conftest_37 not there? (That’s just a text file, so why would that write have failed?) Or does that python executable not exist? (Weird, because I’m pretty sure it worked earlier in the SCons session.) I’m not sure what to make of that. Everything after that that it tries with different possible library options fails with the same error.

So maybe you can look in your .sconf_temp directory at the different conftest_36 and conftest_37 items. Sometimes running the commands yourself from the command line rather than from within SCons makes it clearer what the problem actually is.

1 Like
#13

Could it be libfftw causing the problem? An MKL clash.

1 Like
#14

Now that I have nomkl installed which did indeed pull in the right numpy and scipy… I’m starting a fresh build. I’ll send along the results of that later today… and if there continue to be failures, I’ll try to rerun some of those scons steps at the command line.

#15

Unfortunately, it seemed to fail in the same fashion

bash-4.1$ eups distrib install lsst_distrib -t v13_0 --nolocks
   [  1/93 ]  apr 1.5.2                                                  done.
   [  2/93 ]  cfitsio 3360.lsst4                                         done.
   [  3/93 ]  doxygen 1.8.5.lsst1 (already installed)                    done.
   [  4/93 ]  eigen 3.2.5.lsst2                                          done.
   [  5/93 ]  fftw 3.3.4.lsst2                                           done.
   [  6/93 ]  gsl 1.16.lsst3                                             done.
   [  7/93 ]  mariadbclient 10.1.21.lsst1                                done.
   [  8/93 ]  minuit2 5.34.14                                            done.
   [  9/93 ]  mpich 3.2.lsst1                                            done.
   [ 10/93 ]  python 0.0.6 (already installed)                           done.
   [ 11/93 ]  swig 3.0.10                                                done.
   [ 12/93 ]  xpa 2.1.15.lsst3                                           done.
   [ 13/93 ]  activemqcpp 3.9.0.lsst2+3                                  done.
   [ 14/93 ]  apr_util 1.5.4                                             done.
  [ 15/93 ]  astropy 0.0.1.lsst2+2 ...
             Using externally provided astropy v1.3.
   [ 15/93 ]  astropy 0.0.1.lsst2+2                                      done.
   [ 16/93 ]  boost 1.60.lsst1+1                                         done.
   [ 17/93 ]  mpi 0.0.1+2                                                done.
  [ 18/93 ]  numpy 0.0.3+1 ...
             Using externally provided numpy v1.11.3.
   [ 18/93 ]  numpy 0.0.3+1                                              done.
   [ 19/93 ]  pykg_config 1.3.0+1                                        done.
   [ 20/93 ]  python_d2to1 0.2.12.lsst2+1                                done.
   [ 21/93 ]  python_future 0.15.2+2                                     done.
   [ 22/93 ]  python_mysqlclient 1.3.7.lsst1+8                           done.
   [ 23/93 ]  python_psutil 4.1.0+3                                      done.
   [ 24/93 ]  pyyaml 3.11.lsst1+3                                        done.
   [ 25/93 ]  scons 2.5.0.lsst2+1 (already installed)                    done.
   [ 26/93 ]  sqlalchemy 1.0.8.lsst3+3                                   done.
   [ 27/93 ]  wcslib 5.13.lsst1                                          done.
   [ 28/93 ]  astrometry_net 0.67.123ff3e.lsst1+1                        done.
   [ 29/93 ]  esutil 0.6.0+1                                             done.
   [ 30/93 ]  log4cxx 0.10.0.lsst7                                       done.
  [ 31/93 ]  matplotlib 0.0.4 ...
             Using externally provided matplotlib v2.0.0.
   [ 31/93 ]  matplotlib 0.0.4                                           done.
   [ 32/93 ]  mpi4py 2.0.0+2                                             done.
   [ 33/93 ]  ndarray 1.3.0                                              done.
  [ 34/93 ]  scipy 0.0.1.lsst1+3 ...
             Using externally provided scipy v0.19.0.
   [ 34/93 ]  scipy 0.0.1.lsst1+3                                        done.
   [ 35/93 ]  sconsUtils 13.0 (already installed)                        done.
   [ 36/93 ]  stsci_distutils 0.3.7.lsst1+2                              done.
   [ 37/93 ]  tmv 0.73+5                                                 done.
   [ 38/93 ]  astrometry_net_data 10.0+89                                done.
   [ 39/93 ]  base 13.0                                                  done.
   [ 40/93 ]  lmfit 0.9.3+4                                              done.
   [ 41/93 ]  lsst 13.0 (already installed)                              done.
   [ 42/93 ]  psfex 13.0                                                 done.
   [ 43/93 ]  pyfits 3.4.0+7                                             done.
  [ 44/93 ]  galsim 1.4.1.lsst3 ...
 
***** error: from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.log:
TMV version is 0.73
Using TMV_LINK file: /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/share/tmv/tmv-link
     -L/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/Linux64/tmv/0.73+5/lib -ltmv -lpthread -fopenmp
Checking for correct TMV linkage... (this may take a little while)
Checking for correct TMV linkage... yes
Checking if we can build against Python...
Unable to build a python loadable module using the python executable:
/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/bin/python,
the library name libpython2.7.so,
and the libdir /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/lib.
Nor did any of the known variations on this work.
If these are not the correct library name or location, you can tell scons
the correct names to use with the flags EXTRA_LIB_PATH and/or EXTRA_LIBS.
 
Please fix the above error(s) and rerun scons.
Note: you may want to look through the file INSTALL.md for advice.
Also, if you are having trouble, please check the INSTALL FAQ at
   https://github.com/GalSim-developers/GalSim/wiki/Installation%20FAQ
 
+ exit -4
eups distrib: Failed to build galsim-1.4.1.lsst3.eupspkg: Command:
    source /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/eups/bin/setups.sh; export EUPS_PATH=/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0; (/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.sh) >> /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.log 2>&1 4>/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v13_0/EupsBuildDir/Linux64/galsim-1.4.1.lsst3/build.msg
exited with code 252

The gs.error file again mentions a number of conftest_* files as missing, but they exist and have content, for example:

Output of the command '/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/a
naconda/4.3.1/bin/python < .sconf_temp/conftest_90' is:
bash: /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/anaconda/4.3.1/bin
/python < .sconf_temp/conftest_90: No such file or directory

bash-4.1$ more EupsBuildDir/Linux64/galsim-1.4.1.lsst3/galsim-1.4.1.lsst3/.sconf_temp/conftest_100
import sys
sys.path.append('.sconf_temp/conftest_99_mod')
import check_python
print(check_python.run())

bash-4.1$ ls EupsBuildDir/Linux64/galsim-1.4.1.lsst3/galsim-1.4.1.lsst3/.sconf_temp/conftest_99_mod/
check_python.so

so I tried re-running

bash-4.1$ python < .sconf_temp/conftest_100
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
ImportError: libtmv_symband.so.0: cannot open shared object file: No such file or directory

That library exists, I just don’t have the environment set up as the build does even though I’m in the same session where “eups distrib install” failed. Short of duplicating what I see in gs.error by hand, is there an easy way to get my environment arranged properly so I can run some of these commands without scons?

#16

Do you need lsst_distrib? Can you get away with building lsst_obs and lsst_apps? Then you won’t need Galsim.

1 Like