Special (breakout) Session about Cadence Optimization

ivezic · June 6, 2017, 5:00am

Would there be enough interest for a session about simulated observing
strategies and cadence optimization? We could start with a brief presentation
about ongoing Project efforts and then focus discussion on the community
input (what is needed, how to provide input, etc). If interested, please
respond with specific requests/suggestions if possible (e.g. “I want to discuss
how to use MAF”, “What are the hardware and software constraints for
DDFs”, “what can we do in the Galactic plane”).

sridgway · June 6, 2017, 5:56pm

This is a good high level topic, maybe deserves an early plenary slot.

willclarkson · June 6, 2017, 6:43pm

(My reply to Zeljko moved up to start a new topic thread: “Candidate strategies for mini-surveys”)

willclarkson · June 6, 2017, 11:25pm

Within a session on cadence optimization, I’d suggest a short talk or few along the lines of “solved problems you can address with MAF” or something similar. I think a lot of the difficulties that were hampering early cadence progress have since been solved by updates to MAF. Some examples:

Spatial confusion (I think @knutago or @rblum5 might be good people to approach on this topic, also of course @yoachim );
Dust models within MAF;
Vector metrics

(I think vector metrics are going to be critical to help activating the large subset within the community who may have their own sophisticated, specialized simulation tools but don’t know how to get them to work with MAF output.)

dkirkby · June 7, 2017, 12:45am

I am interested in the following question: given a set of science metrics, how can we best optimize the accumulated SNR per exposure taking into account dust, airmass, scattered moonlight, twilight, etc. This is a difficult constrained optimization problem, but improved algorithms can potentially increase the project’s science reach. Could this session include a discussion of the current algorithms and plans for future improvement? It would be great to also hear from people in the community with relevant experience.

If the goal of the current session is to focus on MAF-related issues, is there interest in a separate session focussed on scheduling & optimization algorithms, and possibly also the simulation of survey conditions?

Tony · June 7, 2017, 8:27pm

@dkirkby I agree this will be a useful session. Science cases which depend on statistical analyses of large samples are compromised by image quality and depth variations across the sky. For years we have been saying that we would suppress that by selectively re-observing bad visits. It will be good to see how this would work in an OpSim run that implements that kind of cadence.

wadawson · June 15, 2017, 5:44am

I am interested in this session. In addition to help creating custom metrics, it would be great to have an overview of the cadence plan timeline. For example, when will the cadence plan be finalized, if the current cadence isn’t good for a particular science case what is the best way to try and change this, etc…

bethwillman · June 15, 2017, 3:46pm

I’m hearing interest in a number of topics related to observing strategy, which would be served by more than one session. I attempt to gather a list below, so that we can synthesize/optimize.

Zeljko is certainly the best person to lead an overview talk. @ivezic or others (e.g. @ljones, @drphilmarshall, @yoachim, @willclarkson, @sridgway, @dkirkby) - who else might be good to design and lead sessions, or parts of sessions, on any of these other topics:

Overview talk highlighting what is currently known, what is the cadence plan timeline and process for ingesting community input.
Technical MAF issues, Hands on help for scientists creating metrics - Information that can help scientists push through past difficulties [this would be a specialized/technical session]
Discussion/presentations on scheduling & optimization algorithms (including dithering strategies, see also Dithering goals and strategies). This session could include workshop time spent on defining different types of observing strategies that might suggested to be tried for WFD (e.g. different types of rolling cadences Rolling cadences).
Workshop on candidate strategies for mini-surveys [see Will’s thread here Candidate strategies for mini-surveys]

dkirkby · June 15, 2017, 6:15pm

I am most interested in the “scheduling & optimization algorithms” aspect, and could help organize this. Another closely related topic that might fit well here is the modeling of observing conditions (seeing, transparency, etc) used for cadence simulations.

ljones · June 19, 2017, 4:25pm

Hi @willclarkson what do you mean by vector metrics? I am wondering if we are thinking of different things … you can certainly write metrics which return more than a single value, but what we have called ‘vector metrics’ are things which are specialized to return a time series of values at each slicePoint (e.g. at each healpix point) – but you can always return a dict from the metric, which would let you return pretty much anything else at each slicePoint.

willclarkson · June 19, 2017, 4:35pm

Hi @ljones - by “vector metrics” I did indeed mean metrics that return an array of values for a given point, not necessarily a time series. For example, it would be extremely useful for a metric to output the run of (say) photometric uncertainty with apparent magnitude. Does such an example already exist among the sims_maf tutorials?

So yes I think I must have misunderstood the term as used within maf. All, apologies for any confusion caused by my misunderstanding!

ljones · June 19, 2017, 11:26pm

No worries! Actually, I was looking for an example like you suggested and I couldn’t find one quite the same. However, if you have a look at this metric:

github.com

LSST-nonproject/sims_maf_contrib/blob/master/mafContrib/transientAsciiMetric.py

from builtins import zip
# Transient metric with input ascii lightcurve
# fbb@nyu.edu, svalenti@lcogt.net

import os
import numpy as np
from lsst.sims.maf.metrics import BaseMetric
from lsst.sims.maf.utils import m52snr

__all__ = ['TransientAsciiMetric']


class TransientAsciiMetric(BaseMetric):
    """Based on the transientMetric, but uses an ascii input file and provides option to write out lightcurve.

    Calculate what fraction of the transients would be detected. Best paired with a spatial slicer.
    The lightcurve in input is an ascii file per photometric band so that different lightcurve
    shapes can be implemented.

    Parameters

This file has been truncated. show original

you can see that it can actually return either a dictionary or a single number (fancy! most just choose one or the other).
The varMetrics.py file has another example of a metric that returns a dictionary:
https://github.com/LSST-nonproject/sims_maf_contrib/blob/master/mafContrib/varMetrics.py

and this one

github.com

LSST-nonproject/sims_maf_contrib/blob/master/mafContrib/PeriodicMetric.py

# Example for PeriodicMetric
# Mike Lund - Vanderbilt University
# mike.lund@gmail.com
# Last edited 3/10/2015
# Motivation: The detection of periodic signals can be examined by using canonical signals and attempted to recover these. However, a more general approach would be to examine the strength in signal that is lost as a result of poor phase coverage.
# This metric calculates the spectral window function for a set of scheduled observations. The largest peak at a nonzero frequency is used as a proxy to quantify how much power is lost to other frequencies. Full phase coverage will result in a value of 1. We refer to this as the Periodic Purity Function.

from lsst.sims.maf.metrics import BaseMetric
import numpy as np

__all__ = ['PeriodicMetric']

class PeriodicMetric(BaseMetric):
   """From a set of observation times, uses code provided by Robert Siverd (LCOGT) to calculate the spectral window function.
   """
   def __init__(self, TimeCol='expMJD', **kwargs):
      self.TimeCol=TimeCol
      super(PeriodicMetric, self).__init__(col=[self.TimeCol], **kwargs)

   def run(self, dataSlice, slicePoint=None):

This file has been truncated. show original

returns an array.

Both of these simply store the metric values at each point in a numpy object array.
The “vector metrics” we have, at

github.com

lsst/sims_maf/blob/master/python/lsst/sims/maf/metrics/vectorMetrics.py

import numpy as np
from .baseMetric import BaseMetric
from scipy import stats

__all__ = ['HistogramMetric','AccumulateMetric', 'AccumulateCountMetric',
           'HistogramM5Metric', 'AccumulateM5Metric', 'AccumulateUniformityMetric']


class VectorMetric(BaseMetric):
    """
    Base for metrics that return a vector
    """
    def __init__(self, bins=None, binCol='night', col='night', units=None, metricDtype=float, **kwargs):
        super(VectorMetric,self).__init__(col=[col,binCol],units=units,metricDtype=metricDtype,**kwargs)
        self.bins = bins
        self.binCol = binCol
        self.shape = np.size(bins)-1

class HistogramMetric(VectorMetric):
    """

This file has been truncated. show original

are more specialized … the dimension at each slicepoint is assumed to be time, the metricValues are stored as a two-d array (instead of one-dimensional object array), and there are some special plotters that can interpret and deal with this information directly.

willclarkson · June 20, 2017, 3:47pm

Thanks very much for clarifying, @ljones ! It’s great to see that maf does have these capabilities.

From the point of view of making sure the community knows that maf has these capabilities, I think it would be useful for this to be demonstrated in some form within a Cadence breakout session.