We need to get a student at Harvard working on calibration up and running with a modern stack. It’ll need to work with obs_decam, and I don’t think v10_1 is new enough.
There are lots and lots of tags – is there a list? What do we recommend? And are any of them available as binary installs (e.g. from conda, or using docker if necessary). Are there any that build against v10_1 e.g. without updating anything like boost, in which case a binary base install and a source on top would be fine.
I do not know of a good general-purpose answer to this question.
However, there are a few “weekly” tags of the form w_20XX_YY (for year XX, week YY). They’re listed (along with a lot of others) at https://sw.lsstcorp.org/eupspkg/tags/. Building the most-recent weekly tag from source is my currently preferred method for keeping fairly well up to date, although it’s hardly ideal.
An additional caveat is that w_2015_33 doesn’t build for me (likely because of the issues with psfex which Nate has since fixed), so I’m stuck on _30 for now.
@frossie builds weeklies on demand so it should be relatively simple for her to generate one next week. 11.0 is also due of course.
Can we have one every week automatically? Or do I need a cronjob to ask @frossie nicely every Sunday morning so she can submit the job during the sermon?
The plan is definitely for it to be automatic. From what I hear it’s close to being that way.
Can we have a manual one every week until it’s automatic? It sounds as if that’s just a sennight or fortnight or so away.
On hipchat @frossie says
I believe Sims people have been using Mario’s anaconda build for more recent binary
but that’s my point. Where do I go to look for details on @mjuric’s anaconda build?
Looks like the latest you get is w2015.30.
Where did you find that w2015.30 (presumably the same as w_2015_30)?
Started with “The LSST stack builds are kept on http://eupsforge.net/conda/dev channel”, followed that and eventually go to https://lsst-web.ncsa.illinois.edu/~mjuric/conda/dev/linux-64/
@RHL, @ktl – @ljones (or I) occasionally build and push out a new version of conda-wrapped binary distrib, depending on the needs of the sims team. It was used during the cadence workshop as an install option (the other one was a docker container that AJC prepared).
I didn’t want to do (or advertise) one for DM so I don’t cause confusion about what’s officially supported and what isn’t (EUPS is the only supported mechanism right now, I think). It shouldn’t be hard, though – both the code and the instructions are on https://github.com/mjuric/conda-lsst.
I could do a one-off manual build of lsst_distrib. For anything more than that (i.e., unattended automated builds), the code would probably need a few more days of cleanups and testing.
If you don’t want to do it, maybe square could? With every weekly tag, maybe?
Yes the plan is to indeed do weekly binaries, and once we can drive those from Jenkins we are happy to produce them for people who have a robust recipe that can be automated (eg. I talked to Fabio about building his CernVMFS images last week).
I think that the binary distribution available via CernVM FS should work for this use case. Currently you will find there the latest official stable version, i.e. v10.1, for both Linux and MacOS X.
You can use the stack as is or build your own packages on top of this distribution (using EUPS). The relevant details are here:
Look at the Advanced Usage section for details on how to develop on top of a version of the binary stack.
This distribution is also available to use via a Docker container. Details: https://github.com/hepsw/docks/tree/master/cvmfs-lsst
The problem is that 10.1 is too old. 11.1 will be out soon, but it’ll soon be superseded in turn; we need a way to get binary distros of at least the weeklies that Frossie produces.
I think frequent releases and frequent binaries are important for some kinds of users, but I’ve become convinced that at least right now most core developers of the science pipelines stack are better off just building everything from source and updating that stack in-place frequently, and what we should be providing are better tools to do that.
Everyone I know who is not building from source (and instead tries to mix-and-match master packages with released versions) seems to encounter a nonexistent bug or segfault (actually just a version incompatibility) at least once a month. Those don’t just slow down the developer who ran into the problem; they distract everyone else who helps try to reproduce and debug the problem. And while frequent releases will make that less of a problem, I don’t think it will go away anytime soon; there’s simply too much churn. Moreover, installing new releases frequently will either force users to delete old releases frequently or deal with the EUPS sluggishness that comes from having too many versions and tags. And on the other hand, I think the idea that developers need to have a lot of tagged versions on hand to reproduce bugs reported specifically in those versions simply hasn’t materialized (at least not yet). Finally, it makes developers self-sufficient: if they can build everything themselves just as easily as they can ask someone in e.g. SQuaRE to do it, they can’t have their work blocked by SQuaRE availability or CI downtime.
A lot of those problems with building against releases are problems that will have to get fixed anyway: lower-level packages will stabilize, EUPS performance will be improved, improved CI automation will reduce human-availability blockage. But we’ve been struggling to solve this problem by having more frequent releases for years now, and we’ve put very little effort into build-the-stack-from-source tools for developers, and essentially abandoned the effort we have done (lsst-build and lsstsw) to “use at your own risk; this was intended for CI”. There might be a faster solution to the new developer problem there.
@jbosch I’m not sure the two are exclusive – wouldn’t you still benefit from a tool that shortcircuits any build for which a binary distribution exists?
E.g., lsstsw could be made smart enough to check if (say) afw-10.1-38-gf63c96a has already been built by Jenkins, and fall back to building from source otherwise. Since versions uniquely encode all dependencies, this should be relatively safe to do (I think…).
I don’t think they’re mutually exclusive at all, and that sounds useful, but it sounds a lot harder to write than a much simpler tool that just built from source. Heck, maybe lsstsw is already there; I’m happy enough using my own tool for this purpose anyway. I’m just not comfortable suggesting either to newcomers, because I know they’re not really being supported or documented for that role.
Yup, I agree – we officially don’t want to recommend anything unofficial 
.
I’d find an official build tool helpful as well. There is a proposal for what lsst-build could evolve into. Maybe we could find some resources to move on that now, if there’s interest.
Android’s repo tool may be something to look at as well. I haven’t looked at it much, but it seems designed to solve our problem.