New install of lsst_stack, numpy not found

Support
#1

Fresh install of lsst_stack.

Running sdss demo and I get this error:

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ ./bin/demo.sh

Traceback (most recent call last):

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin/processCcd.py”, line 23, in <module>

from lsst.pipe.tasks.processCcd import ProcessCcdTask

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/python/lsst/pipe/tasks/processCcd.py”, line 22, in <module>

from lsst.ip.isr import IsrTask

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/init.py”, line 27, in <module>

from .isrFunctions import *

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/isrFunctions.py”, line 23, in <module>

import numpy

ModuleNotFoundError: No module named ‘numpy’

Then

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ conda install numpy

Collecting package metadata (current_repodata.json): done

Solving environment: done

All requested packages already installed.

#2

Did you have the LSST conda environment activated when you ran shebangtron? What’s the first line of /Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin/processCcd.py ?

#3

#!/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin/python

#4

#!/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin/python

I believe I did have LSST miniconda active, yes.

#5

I did this before shebangtron: source loadLSST.bash # for bash

#6

That’s the LSST-installed miniconda, but not the one from the scipipe_conda_env environment, which should be something like /Users/blum/lsst_stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/bin/python

But if you followed these directions:

source loadLSST.bash
eups distrib install -t v19_0_0 lsst_distrib
curl -sSL https://raw.githubusercontent.com/lsst/shebangtron/master/shebangtron | python

I’m not sure how things could have gone wrong.

What is the output of this?

source loadLSST.bash
command -v python
#7

/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin/python

#8

That’s interesting. How about the output of echo $PATH; echo $LSST_CONDA_ENV_NAME?

#9

(base) MacBook-Pro-79:lsst_stack blum$ echo $PATH

/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/obs_sdss/19.0.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/dax_ppdb/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/verify/19.0.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/meas_astrom/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/meas_base/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pykg_config/1.3.0+6/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/daf_butler/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/astro_metadata_translator/0.1.0-5-g1039c3c+8/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/starlink_ast/lsst-dev-gd6cc4e835a/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/gsl/2.6/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/cfitsio/3360.lsst6/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/fftw/3.3.4.lsst2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pex_config/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/afw/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pex_policy/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/apr_util/1.5.4.lsst1/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/apr/1.5.2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pyyaml/5.1+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/python_psutil/5.4.3+4/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/boost/1.69.lsst1/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/doxygen/1.8.13.lsst3/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pep8_naming/0.8.2+3/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pyflakes/2.1.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/flake8/3.7.5+3/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/scons/3.0.0.lsst1+5/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/sconsUtils/19.0.0/bin:/Users/blum/lsst_stack/eups/2.1.5/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/usr/local/git/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/meas_algorithms/19.0.0/bin

#10

and
lsst-scipipe-4d7b902

#11

Ah, something has gotten confused here, and the pair of minicondas at the beginning of the path is another indication of this. You’ll need to start in a fresh shell. The first entries in your PATH after source loadLSST.bash should be /Users/blum/lsst_stack/eups/2.1.5/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin

I guess it might be good to look at the contents of your loadLSST.bash as well.

I’m also wondering if /opt/anaconda3/condabin could be part of the problem, although it shouldn’t be.

#12

In a fresh shell, after source loadLSST.bash the second and third paths are reversed from what you write above.

#13

Ah! I think I may have found the problem (which would be in newinstall.sh). I’ll try to reproduce and then file a bug.

#14

Nope, that wasn’t it, things look fine in my brand new (but not macOS) installation. I will need the output of cat loadLSST.bash

#15

cat loadLSST.bash

This script is intended to be used with bash to load the minimal LSST

environment

Usage: source loadLSST.bash

export PATH="/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:${PATH}"

export LSST_CONDA_ENV_NAME=${LSST_CONDA_ENV_NAME:-lsst-scipipe-4d7b902}

shellcheck disable=SC1091

source activate “$LSST_CONDA_ENV_NAME”

LSST_HOME="$( cd “$( dirname “${BASH_SOURCE[0]}” )” && pwd )"

Bootstrap EUPS

EUPS_DIR="${LSST_HOME}/eups/2.1.5"

source “${EUPS_DIR}/bin/setups.sh”

export -f setup

export -f unsetup

export EUPS_PKGROOT=${EUPS_PKGROOT:-https://eups.lsst.codes/stack/osx/10.9/clang-1000.10.44.4/miniconda3-4.7.10-4d7b902|https://eups.lsst.codes/stack/src}

#16

I did a sudo \rm -r lsst_stack and reinstalled. The reversal of the two paths noted above was corrected, but the result of running the demo script was still the same. Seems numpy must be in the wrong place.

#17

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ conda list |grep numpy
numpy 1.16.2 py37hacdab7b_0
numpy-base 1.16.2 py37h6575580_0

#18

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ ./bin/demo.sh
Traceback (most recent call last):
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin/processCcd.py”, line 23, in
from lsst.pipe.tasks.processCcd import ProcessCcdTask
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/python/lsst/pipe/tasks/processCcd.py”, line 22, in
from lsst.ip.isr import IsrTask
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/init.py”, line 27, in
from .isrFunctions import *
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/isrFunctions.py”, line 23, in
import numpy
ModuleNotFoundError: No module named ‘numpy’

#19

Hi @rblum5, just checking in to see if you found a way to resolve the issue of numpy not being found, or if you’re still in need of assistance on this topic?

#20

Thanks for asking @MelissaGraham!

I have not had time to get back to this and its still an open issue. Important but not urgent.