Building obs_decam

Support
#1

Hello,

I am new in LSST. I have my LSST stack installed with miniconda. Currently I am trying to install the obs_decam packages so that I can pass images from DECam through the stack. But when I was using scons to build the packages after I setup pipe_tasks, I received the error which is shown below. I’d appreciate it if you can help.

Thank you

Leng

Error:
scons: Reading SConscript files …
EUPS integration: enabled
Checking who built the CC compiler…error: no result
CC is gcc version 4.4.7
Checking for C++11 support
Checking whether the C++ compiler worksno
Checking whether the C++ compiler worksyes
C++11 supported with '-std=c++0x’
Checking for C++ header file tr1/unordered_map… yes
TypeError: can only concatenate list (not “tuple”) to list:
File “home/mleng/miniconda2/opt/lsst/decam/obs_decam/SConstruct”, line 3:
scripts.BasicSConstruct(“obs_decam”, defaultTargets=scripts.DEFAULT_TARGETS + (“decam”,))

#2

See: C++11/14 (gcc 4.8) now the baseline

You need to be using at least GCC 4.8.

Try installing devtoolset-3.

#3

I don’t think that’s the whole story, @timj (though it’s true!). The reported error is:

TypeError: can only concatenate list (not "tuple") to list

and appears to be due to concatenating scripts.DEFAULT_TARGETS (presumably a list) and the tuple ("decam",). This seems to be a change introduced by @rowen rather recently.

This probably needs a ticket and a fix.

#4

I thought the same, but: scripts.DEFAULT_TARGETS has been a tuple since it was introduced by @timj himself back in November of last year: https://github.com/lsst/sconsUtils/commit/39bd372694b9a49ab4cb3e0b15c4378fecd5885a

#5

I am actually confused. I don’t think this stack is current. sconsUtils no longer even works with GCC 4.4 and should fail out immediately. The error above indicates the stack is a few months old.

#6

So we probably need to ask @mbl for more detailed instructions on how-to-reproduce.

#7

Hi, the channels I added in miniconda was http://research.majuric.org/conda/stable, from which I installed the stack. This might be an older version of stack.

#8

Hi,

Since I was unable to find a complete tutorial, I tried setting up obs_decam by following the steps to build obs_cfht, which I found here: https://github.com/DarkEnergyScienceCollaboration/ReprocessingTaskForce/wiki/Installing-the-LSST-DM-stack-and-the-related-packages.

#9

The dates of files in that conda channel are from 2015. You are not going to be able to run a modern obs_decam (straight out of github) with that.

#10

Try building a more modern version of the stack by following the install guide at https://confluence.lsstcorp.org/display/LSWUG/Building+the+LSST+Stack+from+Source .

#11

Try building a more modern version of the stack by following the install guide at https://confluence.lsstcorp.org/display/LSWUG/Building+the+LSST+Stack+from+Source .

Careful: that suggests you run

$ eups distrib install -t v11_0 lsst_apps

which will not get you anything substantially newer than the Conda packages. Using -t v12_0_rc1 is probably a better bet.

#12

Hi, thank you for you suggestion.

I did have tried building from the source at very beginning but encountered some issues. One of them was that why it was checking for CC instead of gcc complier? Was this only resulted from the fact that my gcc version was too old?

Thanks

Error:

scons: Reading SConscript files …
EUPS integration: enabled
Checking who built the CC compiler…error: no result
CC is unknown version unknown
Checking for C++11 support
Checking whether the C++ compiler works… no
C++11 extensions could not be enabled for compiler ‘unknown’

  • exit -4
    eups distrib: Failed to build sconsUtils-2016_01.0-4-g49bc714.eupspkg: Command:
    source /home/mleng/miniconda2/opt/eups/bin/setups.sh; export EUPS_PATH=/home/mleng/LSST:/home/mleng/miniconda2/var/opt/eups; (/home/mleng/LSST/EupsBuildDir/Linux64/sconsUtils-2016_01.0-4-g49bc714/build.sh) >> /home/mleng/LSST/EupsBuildDir/Linux64/sconsUtils-2016_01.0-4-g49bc714/build.log 2>&1 4>/home/mleng/LSST/EupsBuildDir/Linux64/sconsUtils-2016_01.0-4-g49bc714/build.msg
    exited with code 252
#13

I think this code is correct. obs_decam builds fine for Jenkins and on my machine and I emulated the following code in obs_subaru's SConsctruct file:

from lsst.sconsUtils import scripts
scripts.BasicSConstruct("obs_subaru", defaultTargets=scripts.DEFAULT_TARGETS + ("hsc",))
#14

Yes. See my original reply. You need to install GCC 4.8 or newer.

#15

Thank you for your reply. I have installed the latest the version of the stack but while installing obs_decam, the same error persists:

scons: Reading SConscript files …
EUPS integration: enabled
Checking who built the CC compiler…error: no result
CC is gcc version 4.8.5
Checking for C++11 support
Checking whether the C++ compiler worksyes
C++11 supported with '-std=c++11’
Checking for C++ header file tr1/unordered_map… yes
TypeError: can only concatenate list (not “tuple”) to list:
File “/home/mleng/lsst/Linux64/obs_decam/SConstruct”, line 3:
scripts.BasicSConstruct(“obs_decam”, defaultTargets=scripts.DEFAULT_TARGETS + (“decam”,))

Any suggestions on ways to fix this is very much appreciated.

#16

I have installed the latest the version of the stack

Could you be more specific about exactly which version this means, and how you installed it (through Anaconda, using the guide @price suggested, or some other method)?

Please set up your system as if you were about to build obs_decam, then try running both

eups list -s sconsUtils

and

grep DEFAULT_TARGETS $SCONSUTILS_DIR/python/lsst/sconsUtils/scripts.py

and post the output here.

#18

Hello jdswinbank,

Thank you for your reply. I installed the version 12 as what you
suggested. I have the stack installed this time from the source. I have
run the test data and it proceeded successfully.

To build obs_decam, after loading LSST environment I setup pipe_tasks
first. I tried the commands you suggested and the followings were the
outputs:

bash-4.1$ eups list -s sconsUtils
11.0 current b1711 v11_0 setup

grep DEFAULT_TARGETS scripts.py
DEFAULT_TARGETS = (“lib”, “python”, “tests”, “examples”, “doc”, “shebang”)
defaultTargets=DEFAULT_TARGETS,
def finish(defaultTargets=DEFAULT_TARGETS,

Thanks for your help.

#19

I installed the version 12

but

bash-4.1$ eups list -s sconsUtils
11.0 current b1711 v11_0 setup

This worries me: version 12 of the stack should not be using sconsUtils version 11.

I suggest starting from a clean environment – ie, remove anything relevant to LSST from your shell initialization files (.profile, .bashrc, etc) if you’ve added it, then open a fresh new terminal. Start by sourcing loadLSST, as described in the instructions @price linked you to earlier. Then check which versions of both lsst_apps and sconsUtils you have installed:

eups list lsst_apps
eups list sconsUtils

I’m confused by the rest of your reply: it looks like you edited it while I was typing this. At one point, it seemed like $SCONSUTILS_DIR wasn’t set to anything useful, which could be the source of your problems, but you’ve now removed the text that indicates that. Can you confirm what the contents of $SCONSUTILS_DIR is (ie, echo $SCONSUTILS_DIR) immediately before you try to build obs_decam?

#20

Hi, here is the update:

grep DEFAULT_TARGETS scripts.py
DEFAULT_TARGETS = (“lib”, “python”, “tests”, “examples”, “doc”, “shebang”)
defaultTargets=DEFAULT_TARGETS,
def finish(defaultTargets=DEFAULT_TARGETS,

I will try removing everything related to version 11 and update the outputs. Thank you.

#21

Hello jdswinbank,

After cleaning everything related to version 11 and after loading lsst environment, I checked the versions of followings:

bash-4.1$ eups list -s sconsUtils
2016_01.0-8-ge052bd8 b2060 v12_0_rc1 b2058 qserv-dev setup

bash-4.1$ eups list lsst_apps
2016_01.0-1-gd7afef2+24 current v12_0_rc1 b2060

bash-4.1$ eups list sconsUtils
2016_01.0-4-g49bc714 sims_2_2_4c dax_2016_03 b1990 current b1962 w_2016_15 w_2016_12 b1997 qserv_2016_03 b1969
2016_01.0-8-ge052bd8 b2060 v12_0_rc1 b2058 qserv-dev setup

bash-4.1$ grep DEFAULT_TARGETS scripts.py
DEFAULT_TARGETS = (“lib”, “python”, “tests”, “examples”, “doc”, “shebang”)
defaultTargets=DEFAULT_TARGETS,
def finish(defaultTargets=DEFAULT_TARGETS,

When I tried building obs_decam I received the following:

bash-4.1$ setup pipe_tasks
bash-4.1$ setup -k -r .
bash-4.1$ scons install declare
scons: Reading SConscript files …
EUPS integration: enabled
Checking who built the CC compiler…(cached) error: no result
CC is gcc version 4.8.5
Checking for C++11 support
Checking whether the C++ compiler works… (cached) yes
C++11 supported with '-std=c++11’
Setting up environment to build package ‘obs_decam’.
scons: done reading SConscript files.
scons: Building targets …
python decam/makeLinearizer.py decam/linearity_table_v0.4.fits --force
Traceback (most recent call last):
File “decam/makeLinearizer.py”, line 13, in
from lsst.ip.isr import LinearizeLookupTable
ImportError: cannot import name LinearizeLookupTable
scons: *** [decam/linearizer] Error 1
scons: building terminated because of errors.

Thanks