Calib construction scripts

Data Management DM Notifications
, ,
#1

In DM-3373 and a few other tickets, we ported over from HSC scripts for constructing calibs. These reside in pipe_drivers, and use the MPI process pool in ctrl_pool. Example command-lines for using these scripts are on DM-5124, but here’s a rough guide:

  • Bin scripts are constructBias.py, constructDark.py, constructFlat.py and constructFringe.py.
  • These are like a regular CmdLineTask, but with additional arguments that configure how to do the parallelism:
  • --batch-type may be smp (Symmetric Multi-Processing, i.e., run on the local machine with multiple cores), pbs (submit the batch request to your PBS cluster) or slurm (submit the batch request to your Slurm cluster).
  • --job is the job name (used by batch systems to identify the job, and in naming output log files).
  • --time is the walltime limit for batch systems, per element (in this case, the element is individual CCDs).
  • --cores (for smp or slurm modes) specifies the number of cores to use.
  • --nodes and --procs (for pbs or slurm modes) specifies the number of nodes and CPUs per node to use.
  • --batch-submit can be used to provide additional options for batch submission (i.e., qsub or sbatch, as appropriate).
  • --mpiexec can be used to provide additional options for mpiexec. On our Slurm clusters at Princeton, we’ve found we need -bind-to socket to prevent trouble.
  • --dry-run can be used to test the command-line before submitting.
  • You will probably need some configuration additions in your obs_* package.
  • The individual scripts write bias_config, dark_config, flat_config and fringe_config datasets, so your Mapper needs to support those.
  • Overrides can be added to your obs_*/config directory as bias.py, dark.py, flat.py and fringe.py.

@merlin is trying to get this working with DECam at the moment. Hopefully we can report success soon! Users of other cameras are welcome to request help here.

#2

Steps taken so far are to splice in:

bias_config: { template: "config/bias.py" python: "lsst.pipe.drivers.constructCalibs.BiasConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit } dark_config: { template: "config/dark.py" python: "lsst.pipe.drivers.constructCalibs.DarkConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit } flat_config: { template: "config/flat.py" python: "lsst.pipe.drivers.constructCalibs.FlatConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit } fringe_config: { template: "config/fringe.py" python: "lsst.pipe.drivers.constructCalibs.FringeConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit
}

to DecamMapper.paf, just so that steps are recorded for others.

#3

@KSK I am also about to try this for the newly created obs_monocam as well :slight_smile:

#4

This now works with:

  • DECam, using branch tickets/DM-5899 of obs_decam; and
  • Monocam, using branch u/price/20160429 of obs_monocam.

Note that these branches will (hopefully!) be cleaned up as part of their journey to master.