Ideas for EUPS how-to documentation

[jsick (Jonathan Sick)]

Continuing the discussion from Possible documentation errors:

Possible documentation errors

Finally - on the documentation front - I think we do need somewhere some basic information about eups, how it works, what setup does, what tags are, what tags and package versions are available to install, what tags and packages are available to set up, and what packages are actually set up. I don’t think we have that available anywhere for users. I think this might be something that SQUaRE can take on perhaps, @jsick? (I have certainly seen a lot of the questions and problems that people have run into, so if you want to collaborate on it, that’s good too).

Yes, I’d love to work on some practical EUPS-for-LSST documentation this summer.

On the reference side, yes, we do need to list tags. Otherwise I think we send people to an EUPS server URL to find tags, which isn’t an ideal solution. We also need to list what top-level packages are available, and why someone might install one or another (see also RFC-305, which seeks to more clearly define their roles).

We also want some EUPS overview documentation (define basic terminology, like tags versus versions, and the philosophy of EUPS in general).

But I think the core of the EUPS-for-LSST documentation should be short how-to recipes for dealing with certain situations.

So I’d like us ot use this thread to write down what things we do with EUPS, and how we identify and work-around issues. I’ll use the results as research material to put together an EUPS how-to for piplines.lsst.io.

To kick things off, some basic tasks are:

• Installing a package, eups distrib install ...
• Setting up a package (and all variations on setting up specific tags, and the -j)
• Creating tags, eups declare
• Listing installed packages, eups list
• Listing installed versions of a package, eups list -v packages
• Listing set up packages, eups list -s

This is just a cursory beginning. If others from DM and Sims want to write down your common EUPS tasks, that’d be fantastic.

[RHL (Robert Lupton)]

I assume you know about the (poor) TeX documentation in the eups source?

You really need to split the eups and the eups distrib documentation! There are reasons why they are in the same package, but they do fundamentally different things:

1. eups (i.e. eups list, setup, …) manage packages installed on a single filesystem.
2. eups distrib clones a eups tree from one machine to another.

[ljones (Lynne Jones)]

Note that we use eups declare for more than just creating tags … it’s also useful for being able to use a git cloned version of a package or for adding a non-standard package (sims_maf_contrib is the only one I’m aware of here, but maybe there are other DM related packages?).

The major problem with this will be separating the simple “I just want to get stuff done and not learn the wrinkles of eups” from all the possibilities.

For example - that becomes relevant with newinstall.sh + eups (but not lsstsw apparently) – using the manifest.remap file + eups declaring system packages!

[ljones (Lynne Jones)]

Yes, I agree making a super clear distinction between “eups distrib” and “eups” is important. I don’t know if the docs have to be completely split, because it’s helpful for people to realize why/how they are related. Plus there are similarities that are useful … e.g.:
eups list sims_maf
eups distrib list sims_maf

… the first gives you the list of your versions of sims_maf locally available (that you’ve installed or declared), while the second gives you the versions of sims_maf that are available to be installed from the remote server. The second problem you run into with those is that you have to do some work to understand the tags and how they translate into/from versions locally vs. remotely.

[lynnej@astro-lsst-01 lsstEups]$ eups list sims_maf
12.0-8-g053ab26+16 b2527 sims_2_3_2_1 current
2.3.3.sims sims_2_3_3 b2599
2.3.3.sims+6 b2664 sims_2_3_3_3
2.3.4.sims b2866 sims_2_3_4
2.3.4.sims+3 sims b2878 sims_2_3_4_1 b2880
lynnej lynnej

[lynnej@astro-lsst-01 lsstEups]$ eups distrib list sims_maf
sims_maf generic 0.2
sims_maf generic 0.2.2+2
sims_maf generic sims_2.2.4maf
sims_maf generic sims_2.2.4maf+1
sims_maf generic sims_2.2.5
sims_maf generic sims_2.2.5+2
sims_maf generic sims_2.2.6

(I trimmed those lists, btw)

[kfindeisen (Krzysztof Findeisen)]

I pretty much exclusively use eups for setting up a package and listing packages (frequently using -s, and occasionally -v). (Though if I knew how to create tags without having to edit .eups/startup.py, I’d use that feature too.) I don’t use eups distrib because I use lsstsw to manage my stack install, so I’d consider the latter “learn the wrinkles” territory.

[RHL (Robert Lupton)]

You can use one tag for “free” without editing .eups/startup.py, namely your user name (-t kfindeisen). The magic line is e.g.
hooks.config.Eups.userTags.append("tmp")
where the hooks namespace appears by magic. The reason for requiring you to declare tags is to minimise typos (I want -t kfindiesen to fail). You can also declare that you want to use other users tags.

[ljones (Lynne Jones)]

Adding documentation on how to use other people’s tags would be useful for users, I imagine.