Please ensure the results of RFCs are clearly documented

Data Management
#1

This came up during the recent stack bootcamp, but I think it’s worth calling out specifically here:

Over the last several months, we’ve adopted (or considered adopting) several changes to our development practices: Python versions, supported compilers, copyright assignment, branching practices, etc. These decisions have been thrashed out at Bremerton and through the RFC process, but often the ultimate result – or the moment at which it is expected to take effect – isn’t clear to those who weren’t actually there in person when the decision was made (or, sometimes, even to those who were).

To take a couple of examples: I myself was surprised by the definitiveness by which new developers were being encouraged to adopt PEP-8 style Python coding standards at the bootcamp (in particular given that RFC-97 is still listed as “proposed”), and RFC-45 (on copyright) currently ends with a plea from @jbosch for more information about what we’re actually supposed to do now.

I’m aware that the Bremerton meeting, the start of the new cycle, the preparation of the release and preparations for the Bootcamp itself have taken a lot of everybody’s time and attention over recent weeks. I’m also sympathetic to the idea that we want to teach current best practice to new developers rather than wasting their time with old techniques, and I know that our current documentation sucks and that @jsick will ultimately produce something better.

However, even given all that, it’s important to maintain a consistent and accessible set of guidelines as to how we actually expect development to proceed today. To that end, if you have proposed an RFC which has subsequently been adopted, I’d urge you to go back and ensure that our (current, Confluence-based) documentation is updated to reflect current best practice.

1 Like
#2

I’ve also run in with this issue (for example, writing new documentation based on what the RFCs mean as adopted).

Here are some things we can/should do:

  1. Make it a policy that when an RFC is adopted, there is a clear statement of what the adopted resolution was.

    I talked to @frossie about this and I think she said we could add a new field in JIRA that we could call RFC As Adopted or the like. Once a resolution is adopted, this field must be populated. During the RFC process, this field could even be the ‘working’ version of the RFC Resolution so that participants have a clear idea of where the RFC is going.

    Another possibility is that the RFC applicant should continuously update the original RFC statement throughout the RFC process based on the negotiations and deliberations. My only concern about this is that it might be nice to freeze the ‘RFC as Proposed’ as a fixed comparison against the ‘RFC as Adopted’ so that the RFC comment thread makes more sense. Perhaps people are more comfortable with JIRA’s history diffing features than I am, though.

    But overall, we need a JIRA text field that ultimately has the RFC as Adopted. That way my doc tools can automatically grab RFCs and integrate them into the Stack Docs or the DM webpage as a human friendly record of DM policy.

  2. We more clearly link JIRA stories to RFCs. That is, when an RFC is adopted, we also make the stories that go into implementing the RFC. That way we have a better, and trackable, contract for getting the RFC implemented. Maybe @timj can comment on whether we’re already doing something like this in practice, but I haven’t seen it.

  3. Also think that a lot of the gentlepeople’s agreements we reached at Bremerton, or similar in-person contexts need to be RFC’d (even if auto-adopted) just for the sake of documentation.

And I’m totally sympathetic to this. My new Docs (in draft) are trying to skip straight to the current best practices as much as possible. I feel quite badly that I didn’t start working here months, even years earlier, so we’re currently in quite a bad state, especially for on-boarding my fellow new hires.

Developer workflows, code standards, and documentation writing guides are what I’m currently writing. I hope I can publish these in the new week or two.

My game plan has been to ignore and deprecate certain Confluence pages as quickly as possible, and instead just write the new Docs. In a lot of cases, I’ve found that the Confluence documentation isn’t even worth copy-and-pasting into the new docs; re-writes are necessary. We’re better off spending my story points writing real documentation than editing end-of-life wiki pages. I know a lot of people might (and do) criticize this approach, but I think this is the best route. Also, please keep in mind that the first month-and-a-half of my work at DM concentrated on the forum and a soon-be-deployed broadcast bot for community.lsst.org. I’m not finally getting into my stride of just writing the docs. The new era is in sight.

But if DM, as a collection of individuals, can take on updating Confluence pages in the interim as small incremental edits, then I’m all for that. I just can’t be the one to take on that task entirely.

#3

+1 on this – it’s definitely not my intention to suggest that this is a problem @jsick is directly responsible for. I’m not berating him for not providing us with new documentation, I’m berating everybody else for not keeping the current stuff up-to-date.

#4

(and oops, I didn’t mean to suggest you were. I just wanted to express that I do feel the urgency of docs as blocker on a lot of people’s work, and I do have my best intentions here even if some people, not you John, disagree with my approach to not fix Confluence first).

#5

Hear hear. I intended for every RFC to clearly state what action would be taken as a result, but this has not always been enforced.

The resolution of some RFCs has involved a sequence of actions (e.g. an automated tool needs to be developed before something else can happen). This should be (but has not always been) captured in a JIRA ticket. Also, this leaves the RFC in limbo until the JIRA ticket has been completed and closed, which is less than ideal.

I was hoping to find some time to scrub through the current list of RFCs and update documents or drive concluding activities accordingly. I would appreciate any help that others can provide with this.

#6

My JIRA-fu is weak, but is there a way we could give RFCs additional states so they could have a lifecycle of

  1. Proposed
  2. Adopted (planned)
  3. Adopted (in progress)
  4. Adopted

where

  • Stage 2 happens when an RFC is agreed on, but there are open JIRA stories that are linked to the RFC (via the is triggered by JIRA link).
  • Stage 3 is automatically triggered when some work has been registered against the RFC, but not all stories are closed, and
  • Stage 4 is automatically triggered when all stories linked to the RFC are closed.

If we can have this sort of metadata properly implemented in JIRA (along with a well-defined place in the RFC ticket where the RFC language as adopted is written) then I can ‘easily’ write a tool that pulls from the JIRA API and makes a nice table of RFCs with their titles, adoption status, and language that can be published on the DM website and/or the Stack Docs.

#7

I’m not sure we need the distinction between steps 2 and 3 (though I’m not opposed to having both either).

Aside from that, I like this idea a lot.

1 Like
#8

How about Adopted means that we have all agreed that it should be done and use Implemented to indicated that the RFC outcomes have been completed (Whether that is updating documentation or changing code or whatever).

I do like the idea of seeing JIRA tickets directly linked to the RFC (and I have tried to do that when I notice).

2 Likes
#9

I like @timj’s and @jbosch’s suggestion. The lifecycle then would go

  1. Proposed
  2. Adopted
  3. Implemented

where

  • Stage 2 happens when an RFC is agreed on, but there are open JIRA stories that are linked to the RFC (via the is triggered by JIRA link).
  • Stage 4 is automatically triggered when all stories linked to the RFC are closed.

Shall we RFC this lifecycle+adding a JIRA canonical field for the agreed upon text of the adopted RFC? Is the lifecycle monitoring something that can be done automatically, or do we need to write a bot against the JIRA API?

#10

you have to be a bit careful with race conditions if it’s automated because in many cases the JIRA implementation tickets may not have been written at the point where the person changes the state to Adopted.

1 Like
#11

I think it makes sense to make adding the implementation tickets a precondition for adopting an RFC.

1 Like
#12

I agree, with a minor caveat: we should always have at least one ticket that an RFC permits, but we may not know all such tickets in advance. For instance RFC-100 is a request to move from boost:: to std:: where we easily can, but I don’t know all such cases.

#13

I don’t think there are serious objections to any of this; you could just go ahead and do this.

#14

I disagree: I’d prefer for adopted RFCs to be specific enough to have the implementation tickets all identified at time of adoption. If that means we sometimes have to do some exploratory work on another issue before we can adopt some RFCs, so be it, but I don’t want to see a lot of partially-but-not-fully adopted RFCs lingering in the queue.

#15

This is essentially what the LSST change control board ticket process does, too. I like it.

(Discourse note: the list-numbering was not preserved by the “quote in reply” button.)

#16

So I don’t have a huge objection to this, but nevertheless I would like to see a very clean separation between RFCs (which are exactly what it says, a request for comments) and what are essentially work tickets. The obvious scenario is that I can RFC that the SQuaRE Technical Manager should be rewarded for every deployed service with chocolate, but even if it passes without objection (because everybody is thinking “I don’t object if other people bring her chocolate”), I can’t actually force anybody to bring me chocolate via RFC or, indeed, any other mechanism. RFCs belong to all, but effort belongs to the T/CAMs.

If anything, I would like to see the link go the other way around - work tickets that say “bring X to RFC”, where the RFC is the product of the work, not the cause of it.

Just my 2 cents.

#17

The point is well taken. So perhaps all we need is a workflow of

  1. Putting a summary of the RFC consensus into a standardized JIRA field so that it’s easy to understand the final resolution without re-parsing the entire comment thread, and
  2. Having a culture where JIRA relations are made between DM tickets and RFCs so that it’s straightforward to discover the effort related to each RFC (whether it was prompted by, or led to, the RFC).

The end goal, for me, would be to scrape all of this JIRA information into easy-to-read pages in the documentation so we can see exactly what the group decisions are.

#18

Like Frossie, I don’t really object to the procedures being discussed here, but I worry a little that our (perfectly understandable) love for tooling obscures the fundamentals. Picking (unfairly, perhaps) on one line from the above:

Why is this useful? I want to be able to load the coding standards documentation and see how I’m currently expected to write code. It’s not of any practical importance that those standards were arrived at via an RFC process. [To the extent which it is relevant, we should be able to look at the version control history of the standards and see a note to the effect of “adding rule following RFC-XYZ”.]

To approach the problem from a different angle, let me suggest the following: an RFC itself, even an adopted one, does not represent project documentation. If we adopt an RFC to use some particular version of Python, that simply grants permission for the proposer of the RFC to update the documentation to that effect. At all stages, the published documents are canonical: until they have been updated, old procedures and standards remain in force, regardless of the results of the RFC.

This pushes the problem of scheduling work firmly onto the T/CAMs: the fact an RFC has been adopted does not oblige the T/CAM to schedule effort to carry it out, it simply grants them permission to do so at their discretion. If another part of the project is blocked until the work enabled by the RFC has been carried out, they can work with the T/CAM to resolve the blockage just as they would for any other dependency.

(None of this, of course, is to disagree with the point that a clear specification of the work that needs to be carried out to implement an RFC is a prerequisite for the RFC being adopted: that’s true, but orthogonal to a desire for additional statuses or complex workflows.)

1 Like
#19

I think this really boils down to whether the the state of an RFC should reflect a difference between:

  1. We agree that the RFC is good and the associated tickets are for implementation.
  2. All the work associated with the RFC has now been completed.

I think we all agree that adopting an RFC without a clear set of tickets describing the actual work is a bad idea (and the Python 3 adoption RFC is a good example of where that didn’t happen and I’m still struggling with that).

#20

The motivation for this sentiment was that as a new hire, I found RFCs to be relatively hidden in JIRA. But I retract my wish to make a transparent RFC index now that I’ve levelled up my JIRA and realized that the filter project = RFC ORDER BY key DESC + list view is how I’d like to see the JIRA RFC page by default.

Ironically this is exactly why Jim’s original post resonated with me. :slight_smile: I was working on documentation for how Stack packages should be structured/documented. By accident, certainly not intention, I heard about RFC-45 (copyright policy) and that it was Adopted. Naturally I wanted to incorporate that policy into the documentation. However, doing so involved me going through the comment thread to see any changes to the original proposal had been made. So in all, it doesn’t feel like a robust RFC implementation process.

However, I’m willing to admit that it probably is my job to comb through the RFCs and make sure they’re reflected in the documentation. But I also acknowledge that once the backlog of RFCs are actually documented, and a real documentation platform exists, then T/CAMs can create documentation stories in conjunction with RFC adoption. The problem will solve itself.

So after this entire belaboured topic, I’m willing to back off making any change to the transparency of RFCs. T/CAMs have it under control is the message I’m hearing.