Preparing the DM Update for Scientists Session

Data Management
#1

Hi everyone (but (for now) especially @nidever, @RHL, gpdf, @jsick and @ktl ).

There’s a session I’m leading in Bremerton next week on Thursday, called “DM Update for Scientists”. There are several goals:

  • (Re)introduce what LSST DM goals and what data products we’re going to produce (including pointers to where they can find out more)
  • Introduce the (new) DM team members to the science collaboration community, especially those who will be interacting with them the most (e.g., @jsick, @nidever, Colin Slater, etc.)
  • Give an update about new developments since last year, and plans for the following few years
  • Leave some time for discussion, Q&A

Given that, here’s what I roughly have in mind:

  • [10 min?] Introduce new people
    • (maybe also mention new organization?)
  • [20 min?] DM Overview + dev plan (MJ)
  • For planning, I’d re-use much of the planning talk KTL will give on Tuesday
  • [10 min]? DM software/algorithm highlights in new release (rhl? @jbosch?)
  • [10 min?] Data we plan to process (DavidN?)
  • [10 min?] SUI plans (gpdf? xiuxin?)
  • [10 min?] New communication tools (jsick?)

Questions for all of you: this is our opportunity to tell the scientific community what we feel they should know, and also for them to ask us questions and/or provide feedback. Is there anything else we really want to tell them? Or is any of the topics above not yet ready to be discussed?

PS: This is just a 0th draft (that I’ll be updating tomorrow). But comments welcome!

Preparing the DM Status Update Session
#2

I have some slides from the recent HSC meeting on algorithm updates, focusing on the new multi-band coadd processing scheme, which I think could easily be adapted for this.

How do I comment on a reply rather than appending to the thread?
#3

I’m not giving a planning talk on Tuesday. I’m giving a status talk with a slide or two about short-range and long-range plans. So don’t expect much for the second bullet…

#4

Sounds good to me @mjuric. I’m happy get some face time so that folks have a point of contact to raise usability issues, among other things. In the last minutes I’d be happy to get people on community.discourse.org (pending RFC notwithstanding). I’ve talked to Willman and I may also talk about this site at either LSST Communication II Tuesday 11 am - 12:30 pm or Informal Communication Lunch discussion Wednesday lunch 12:30 - 1:30 pm. A bigger audience could really help boost adoption, though.

#5

Throwing @nidever in at the deep end!

Are you just referring to Discourse here or do you have a larger scope for this?

Gregory doesn’t have an account here yet but @xiuqin does.

#6

@mjuric We talked about this with Beth last week, we’re happy to help but can you liaise with her to make sure there’s no duplication/overlap/mixed-messaging. She’s spearheading scence communications issues for LSST it seems.

#7

@mjuric I will be happy to support this. How much detail do you want?

#8

I’m happy to talk about “data we plan to process” as long as we can meet earlier in the week to discuss the plan. :slight_smile:

1 Like
#9
  • @jbosch : yes, that would be great. I’d stick to astronomically obvious functionality as much as possible (i.e., multi-band coadds == go, refactoring of module X == pass)
  • @ktl : OK, I can prepare a brief summary (I’ll go over it with you).
  • @jsick, @frossie : I think we can use this to summarize where the project is going (if a clear direction crystalizes from Beth’s meetings over the past three days), as it pertains to DM. To be honest, I was initially thinking to use this to solicit feedback and/or further testing by the community for things like Discourse. Any other plans SQuaRE may have that impact the community, this may be a good forum to ask about it (e.g., software distribution plans, etc.)
  • @timj, @nidever : No worries, we’ve got resuscitation crews on hand :). On a more serious note, yes, we should meet before to discuss benchmark dataset plans. I’ll set something up for Mon, Tue, or Wed (may have to be over lunch or dinner)
  • @xiuqin: If we’re ready, it would be good to give the community a flavor of what our data access website may look like or what kind of functionality it may support (and invite feedback). Note the “if we’re ready” qualifier! If it’s better to hold off on doing this for a few more months until the plans for “version 1” crystalize, we should do that. We can chat offline.

PS: I’m worried the list of things we wanted to present is too ambitious for the time allotted (1.5 hrs) – we may need to cut something :frowning: .

#10

Hi all (esp. @jbosch, @nidever, @davidciardi, @jsick),

Just to confirm to everyone we’re sticking with the structure laid out in the opening post of this topic. The times listed there do not include the time for discussion. Also, if there’s any time left in the end, we’ll use it for an open Q&A session.

PS: @davidciardi will try to deliver the talk via videocon; if that doesn’t work out, @xiuqin, could you do it (or nominate someone someone from the SUI team to take over)?

PPS: If you want to dry-run (or have me look at) your slides, happy to do that tomorrow evening.

Thanks!!

#11

OK, I will do it if there is any issue with David’s remote delivery. @mjuric @davidciardi

#12

@mjuric Is a talk in the same spirit as the one I gave in ‘Communications I’ ok? Essentially:

  1. introduce my role,
  2. demo Discourse,
  3. show slides inviting astronomers to join us and to think about adopting it for their own purposes.
#13

[10 min]? DM software/algorithm highlights in new release

One suggestion from a new-to-DM, science perspective is that it would be nice to have a sort of “checklist” of features that an astronomer might expect from a photometry package (not just a survey pipeline) that have or have not yet been implemented. E.g.,

  • PSF photometry - Done
  • Coadd creation - Done
  • Moving object [something something something] - ??
  • Deblending - ??
  • Extended source photometry - ??

Overall I’m thinking if-this-was-sextractor, what would your feature list be right now, what is semi-complete, and what is the near term planned feature list. It’s definitely something that would help me understand the current capabilities of DM better.

1 Like
#14

@ctslater: “Done” is a somewhat problematic term. I prefer to think of things as getting to a minimally functional state, then a scientifically useful (state of the art) state, then a meets-the-SRD state. I’d like to eventually get to a color-coded chart that shows our progress on each component; I started working on that but got bogged down in tool issues.

1 Like
#15

Hi everyone,

I have slides from @nidever and @davidciardi, I’m working to finalize my talk, and using the opportunity to ping @jbosch and @jsick to make sure they have theirs ready by ~9:30 am tomorrow :).

#16

I’d love to see that list (or color-coded chart) as well.

#17

Here are my slides; sorry I didn’t send them earlier. I’ll mention a number of changes we’re bringing over from the HSC side, then focus in on the new multi-band processing, which is both the biggest change (IMO) and the one for which I have the best slides.

#18

I think this would be very useful, and I’d be happy to put some effort into it myself - but it won’t happen by tomorrow, and probably not this week. Remind if I forget (unless @ktl or @mjuric want to take the lead on it).

#19

@timj has already been doing some work at https://confluence.lsstcorp.org/display/DM/Data+Processing+End+to+End+Testing and the pages under it which he and I have used to start working on some diagrams. But we need to fix problems with them and then get to coloring.

#20

Thanks everyone,

Here are the presentations:

You have ~40 minutes for a few final updates :smile: @davidciardi – We’re looking into setting up the videocon system in the ballroom; should have more for you in ~15-20 minutes.