This post is in response to a specific request from the opsim team, to handle a slightly more complicated situation, but their request can be broken down into this step + another post (TBD). I figured this may be of wider interest.
Note that a similar idea is also covered in Up and running with sims_maf_contrib and the conda installation is also covered – with a video! – in
Installing LSST stack binaries with conda (screencast).
The basic task here can be broken down into three steps:
-
How do I install the LSST simulations stack (lsst_sims) using conda?
Up to date instructions are currently available at https://confluence.lsstcorp.org/display/SIM/Catalogs+and+MAF
This boils down to (at the moment):
- Install miniconda http://conda.pydata.org/docs/install/quick.html or the full anaconda distribution https://www.continuum.io/downloads
* If you already have an anaconda installation, and want to keep using that installation rather than a new copy of miniconda, you may want to set up a separate environment to install the LSST software into. The LSST software requires particular versions of numpy, scipy, etc., which you may want to keep separate from your existing environment. This is optional! More information about conda environments is available at http://conda.pydata.org/docs/using/envs.html. From here on out, I will assume that if you used an environment, you will take care of the appropriate environment activation and any substitutions in file locations, noting that instead of installing into ~/anaconda/bin, the eups-setup.csh script will be in your ~/anaconda/envs//bin directory.- Do NOT use a version of miniconda or anaconda that has previously been installed by LSST’s
newinstall.shscript. This will lead to conflicts. - Check that this conda installation is now first your path – the results of
which condashould point to the bin directory of your mini/ana-conda.
- Do NOT use a version of miniconda or anaconda that has previously been installed by LSST’s
- Even if you just installed it, do an update to get the most recent version of conda and other packages:
conda update conda
conda upgrade --all
- Add the conda channel for LSST simulations. This adds the LSST sims channel to your ~/.condarc file. If you need to remove this channel later, simply remove that line from your ~/.condarc file.
conda config --add channels http://conda.lsst.codes/sims
- Conda install the lsst-sims (or other specific package, such as lsst-sims-maf). You can also, optionally, install lsst-apps, which provides more of the LSST DM stack than the minimum necessary for Sims. If you want to see which specific git tags are included in a particular conda version of these packages, you do
conda search lsst-simsandconda search lsst-apps.
conda install lsst-sims
conda install lsst-apps
- When necessary, you can update your conda installation of lsst-sims (and lsst-apps, if you installed that too) by simply doing
conda upgrade lsst-sims
conda upgrade lsst-apps
-
How do I use my conda installation of the sims stack?
- Look for the eups-setups.csh (or eups-setups.sh) file in your conda installation. It will be in $conda_dir/bin (and should be in your path, but with tcsh I’ve had some bugs just sourcing the file without the full pathname).
- Source that file - this sets up “eups”, the LSST package manager. If you use tcsh, this means:
source $conda_dir/bin/eups-setups.csh
- Tell eups to “setup” the lsst packages you want to use (
lsst_sims,sims_maf,sims_photUtils, etc.), which makes them active in your environment. Without this step, conda still doesn’t know about the package you’re trying to use. Note that the packages are called things likelsst-sims-mafin conda, but eups still knows them by their “LSST names” ofsims_maf.
setup lsst_sims
You are now ready to use the versions of the LSST simulations packages installed by conda!
Okay - but now, you want to use a version of a package which has come from a git repo instead. Maybe you want to do this because you want to use the ‘master’ or a branch version of sims_maf (or other simulations package), as there were bugfixes which you need. This is simple with the sims packages, which are pure python. Maybe you want to do this for one of the LSST-associated packages which are not conda-distributed, such as sims_maf_contrib or syseng_throughputs. This is also simple, and you can use the same procedure.
-
How do I use a git repo version of a simulations or LSST-associated python package?
- For clarity, I’m going to run through two examples: using the LSST-associated package sims_maf_contrib and using a git (master) version of sims_maf.
- Set up your sims stack for use, as described above.
- Example 1: Get the sims_maf_contrib package installed (an LSST-associated package, but not a replacement for an LSST package). You only have to do this step once!
- In a separate directory – such as ~/lsstRepos – git clone the sims_maf_contrib repository. Then go to that directory, tell eups about it and mark it as “current” (which means it’s the default version to use).
git clone https://github.com/LSST-nonproject/sims_maf_contrib.git
cd ~/lsstRepos/sims_maf_contrib
eups declare sims_maf_contrib git -r . -c -t $USER
- To update your package, simply do a git pull in the relevant directory.
cd ~/lsstRepos/sims_maf_contrib
git pull
- Use the new package.
setup sims_maf_contrib
- Example 2: Get the git master version of sims_maf installed (a replacement for an LSST stack package). You only have to do this step once!
- In a separate directory – such as ~/lsstRepos – git clone the sims_maf repository. Then go to that directory, tell eups about it. You may not want to mark it as current, unless you really do want to default to using this always, so note the difference in the eups declare step here (no -c).
git clone https://github.com/lsst/sims_maf.git
cd ~/lsstRepos/sims_maf
eups declare sims_maf git -r . -t $USER
- To update your package, simply do a git pull in the relevant directory.
cd ~/lsstRepos/sims_maf
git pull
setup -r .
scons
(The ‘scons’ step runs the unit tests, adds the version information, and copies scripts into the ‘bin’ directory. This gets your package ready to use).
- Use the new package. You only have to do the ‘scons’ step after each git pull, or if you change the branch.
setup sims_maf -t $USER
- If you actually want to use a branch instead of master, just go to the sims_maf directory and checkout the branch. If you’ve already done ‘eups declare’, you do not have to repeat that step.
cd ~/lsstRepos/sims_maf
git pull
git checkout my_exciting_bugfix_branch
setup -r .
scons
- and then use it - just as before.
setup sims_maf -t $USER
And you’re set! You are now using the specific packages you setup – in this case you would have sims_maf_contrib as an available package, you would be using the git repo version of sims_maf, and the rest of the LSST sims packages would be the conda-installed versions.
Bonus: if you only want to temporarily use a git repo version of a package, you can simplify the above steps even further. The commands below will set up a git clone version of sims_photUtils, but eups will not ‘remember’ anything about it after you exit this shell because you did not do the eups declare step.
cd ~/lsstRepos/
git clone https://github.com/lsst/sims_photUtils
cd sims_photUtils
setup -r .
You can double-check which versions of which packages you have set up by looking at the output of
eups list -s