Unified Simulations library

Archive Simulations
#1

One of the discussions planned for the December 2015 Simulations Face-to-Face in Seattle is creating a unified library of “generally useful astronomy methods” for use by OpSim, CatSim, and MAF, so that the packages can all be self-consistent when doing things like converting RA, Dec into Alt, Az. The idea is that OpSim, CatSim, and MAF will be distinguished by the classes that they define (i.e. a scheduler, catalogs, and metrics, respectively) but that the physical and geometric calculations underlying those classes will reside in a single, unified package.

In order to have this discussion, we need to compile a list of what we mean by “generally useful astronomy methods.” In my mind, this includes:

transformations between different coordinate systems (equatorial, galactic, horizontal, etc.)

transformations between time systems (International Atomic Time, Universal Time, Dynamical Barycentric Time, etc.)

transformations of telescope parameters (e.g. going from a pointing RA, Dec and a rotTelPos to a rotSkyPos)

calculations of moon and Sun alt, az

calculation of m5 and photometric signal-to-noise from sky brightness/sky SED

calculation of solar system orbits

Any other thoughts?

#2

It would be great to be able to get rid of pyephem since I’ve had to use it in a few places for MAF and the sky model. I think if we had ecliptic coordinates and sun/moon rise/set times I could do it.

#3

I’d also like to dump some healpix functions in there since I use them all over the place. Mainly hpid2RaDec and raDec2hpid and the healpix equivalent of matplotlib.hexbin.

#4

As the author of PAL/palpy (which can do most of the stated calculations), and at the risk of losing my user base, I guess there should be serious consideration of simply adopting astropy.time and astropy.coordinates.

Do you do WCS transformations?

#5

I would vote to keep the moving object ephemeris calculations separate from this other package. Generally in order to make the ephemeris stuff work accurately and fast, you’re going to have to use a fortran third party package. Adding a dependency on this in a package that would be a “base package” for most of sims could be a bad idea as it raises the bar in terms of requirements quite a bit. (i.e. macosx does not come with a fortran compiler by default).
If we write our own ephemeris generation package this becomes a different consideration, but at the moment anything we have will have an underlying requirement for fortran.

Is this basically talking about combining sims_photUtils (photometry) and sims_coordUtils (astrometry) and sims_utils (time) and add additional astronomical sky calculations for moon and sun position, as well as the healpix stuff? Would we then separate any camera-dependent utilities into a separate package (so that this base sims package does not automatically drag in the entire DM stack?)?
I know I may be beating a dead horse here, but I would really like to keep super basic functionality, like calculation of SNR and m5, separate from needing to install the full DM stack.

#6

Q. @danielsf Consistency across sims components can be implemented by using a common interface to underlying software (internal or external) doing the heavy lifting, and so does not necessarily require implementing the calculation itself for example of solar system orbits. So, do you mean having such wrappers/interfaces or were you actually thinking of actually calculating stuff like solar system orbits?

It would be nice to discuss
a. flux calculation in general (what kind of multiple flux/mag calculations) we support
b. Dealing with spatial/temporal/airmass dependence of bandpasses in this

#7

Looking at this from the SOCS point-of-view, like @ljones I also vote for a package that relies on as little of the stack as possible. I’ll also second @timj’s point of using existing tools when we can. sims_operations currently relies a lot on palpy for astronomical sky calculations. I think most items that we’ll need have been covered, but I’ll list them out for completeness: time conversions, sky coordinate conversions, sun, moon and planet positions and distances on sky, sunset/sunrise/twilight times, moon phase, airmass. M5 and seeing calculations might be useful to have in a central library as well.

#8

It should be pretty easy to do all of this without bringing in unnecessary reliance on the rest of the stack. The CatSim code has already been refactored so that the only package that depends on afw is sims_coordUtils and anything that depends on it. Furthermore, sims_coordUtils now only contains camera-specific functionality. All of the astrometry calculations that only rely on PALPY now live in sims_utils (which means that maybe we should consider renaming some of the packages, or spinning off sims_cameraUtils as its own package and moving PALPY-only astrometry back to sims_coordUtils).

For the purposes of this discussion, I am agnostic as to whether we implement our own methods or wrap third-party methods (though anyone who has met me knows I am habitually suspicious of depending on third party packages; I’d rather be able to see what is going on under the hood, in case anything goes wrong). This discussion is (to my mind) about making sure that CatSim, MAF, and OpSim all do the same thing wherever appropriate. I think we are currently in a state where (for example), CatSim and OpSim each have their own routines to go from RA, Dec to Alt, Az. The two methods probably agree, but no effort has been made to verify that.

#9

@timj CatSim only deals with WCS-es in sims_GalSimInterface, and that is only for the purposes of adding them to the FITS images created by GalSim.

#10

I also do not think we need to have one base library to rule them all. We can have a set of base packages (i.e. sims_coordUtils for astrometry; sims_photUtils for photometry; some new sims_ephem for ephemeris calculations) so long as we all agree to use them where appropriate.

I see this discussion as being more about making sure that things are not implemented twice, rather than about making sure that they are all implemented in the same place.

#11

@yoachim ecliptic coordinates and sun/moon/rise times should be imminently doable. As Tim said, I think PALPY already does those things.

#12

yes, although Sun/Moon/rise times needs some coding on top of palpy (palpy knows where the Sun and Moon are but doesn’t have an explicit rise time function). My perl implementation is here:

#13

The following is the result of the discussion we had on December 16, 2015 about creating a set of unified sims library packages to provide basic functionality required by CatSim, OpSim, and MAF. I have sub-divided the functionality into what I think are logical categories, each representing a hypothetical separate eups package. This can, of course, be reorganized if dependencies become too complicated. The hope is that we will be able to implement all of this with minimal dependence on compiled packages from the DM stack.

Site parameters

OpSim, CatSim, and MAF each have structs which they use to store data about the LSST site (CatSim allows users to specify other site data if they want). We should adopt a single formalism for this.

Astrometry

The coordinate functionality we need to support is:

  • Transformation between equatorial, galactic, ecliptic, and horizontal
    coordinates (including calculating parallactic angle)

  • Transformation from RA, Dec to healpix coordinates

  • Calculation of airmass (mostly, we just need to choose a single convention
    for what we mean by “airmass”; i.e. are we just using sec(Z)?)

  • Transformation between different telescope orientation parameters
    (i.e. going from rotSkyPos to rotTelPos). Currently, the principle
    ambiguity here is whether OpSim uses the beginning or the middle of
    an exposure when calculating altitude, azimuth, and parallactic angle
    for converting from rotSkyPos to rotTelPos. See JIRA issue SIM-1091

  • Tracking the altitude and azimuth of the Sun, Moon, and planets and then
    converting those coordinates into rise and set times.

    • OpSim currently has classes specifically designed to track the positions
      the Sun and the Moon. We should evaluate whether we want the rest of
      sims to simply adopt these classes.

    • We also need to keep track of Moon phase.

  • Transformations between time scales (TAI, UTC, TDB)

  • OpSim requires a method to store the positions of bright stars so that
    the SOCS knows when it is pointing at them. This method may need to be
    able to correct those positions for proper motion, parallax and radial
    velocity. There will be 1000s of stars in this catalog.

Most of these feature are provided in PALPY (though, as Tim Jenness has pointed out, we would have to add some significant code to go from Sun/Moon alt, az to rise and set times). They are also provided in astropy. We must decide whether or not we want to write our own thin wrappers to PALPY, or officially adopt the astropy.coordinates framework. There is some concern that astropy.coordinates may not yet be as fast as working with numpy arrays of floats. An advantage of astropy.coordinates is that it automatically handles angle units. We would no longer have to carry around two copies of each method (one for radians; one for degrees).

It should be possible to implement this functionality in such a way that its only external dependency will be either PALPY or astropy (or both). We may need Healpy to implement the (RA, Dec) to healpix transformation, but that can be implemented in a separate package, if we want.

Photometry

Photometric quantities that we need to be able to calculate are:

  • An official, sims-wide method for calculating m5 from sky brightness.

  • Official methods for calculating FWHMeff and FWHMgeom.

  • Official wrappers for Peter’s sky brightness model to help calculate
    when twilight begins (for the purposes of OpSim).

  • We should determine a method for keeping track of coorections to filter
    throughputs due to airmass and/or position on the focal plane.

We should be able to implement this functionality in a pure python package
without any external dependencies.

Orbits

NEO science will need methods to propagate orbits through time. This will need to be a totally new package, as it will draw in dependencies (pyorb/oorb and fortran at least) that nothing else in the stack requires.