Pinning conda packages installed with DM (lsst_apps and lsst_sims)

Support
#1

I’m trying to pin the conda packages associated with my DM installed from source using newinstall & eups distrib install. I created the pinned file using the results of conda list --export, so I am specifying that I want an exact match, rather than wildcarding. When I go to do something like conda install basemap for example, I receive an error message:

Solving package specifications: .                                                 
PackageNotFoundError: Package not found: '' Packages missing in current linux-64 channels:```              

Then I ran across this JIRA:
https://jira.lsstcorp.org/browse/DM-9074
Should I be pointing to another channel that may perhaps have a stable set of DM specified packages so I can avoid these types of errors?
#2

Pinging @timj and @josh in case they have some suggestions.

#3

Was there more to the error message? Is the failure consistently repeatable? I believe we’ve seen occasional “hiccups” with the public conda channels.

#4

I played some more with the pinned file and that problem has disappeared :slight_smile: Likely my syntax was wrong.
now, when I go to install an additional package I get an error like

conda install basemap
Fetching package metadata .......
Solving package specifications: ....

UnsatisfiableError: The following specifications were found to be in conflict:
  - basemap
  - numpy 1.11.2 py27_nomkl_0 -> feature:nomkl
Use "conda info <package>" to see the dependencies for each package.

What I want to do is have my conda root environment pin DM-related packages, and allow interested users to add more python packages to the same environment. That seems to be very problematic in conda-land, unless one doesn’t worry too much about upgrading packages like numpy, scipy, etc. In particular, I’ve had to be careful to avoid pulling in mkl versions of those packages. If I remove the pinned file, I can do conda install basemap nomkl and that mostly works, but conda desires to upgrade numpy and scipy and I was hoping to avoid that. With the pinned file, I get errors like I mention at the beginning of this reply.

Thanks,
Heather

#5

Conda’s constraint resolution seems to be pretty standard for a package manager. Keep in mind that many of the conda packages DM is pulling in contain compiled object code that has been linked to other packages, so an update of a single package may pull in a large set of dependencies.

It might be worth seeing if the --no-update-dependencies flag to conda install is helpful for your use case. It may also be worth loosening the version constraints on the “patch” version to see if that helps resolution. Eg., https://conda.io/docs/using/envs.html#create-environment-file-by-hand