RSE Wiki Archive

Please see github_auth48_experiment for context.

Summary:
This was the first time I've had such a long document go through the AUTH48 process, but from my perspective things worked much better than they would have using email.

There were literally almost 100 separate changes to the text during AUTH48 (not sure if this is typical or not for this size of document). Being able to track each of these individually, set the appropriate status for each issue, distribute the work across the authoring team and Jean, bring people in to comment on specific issues, and finally review proposed changes with full diffs and ability to easily discuss the diffs - each of these was a significant efficiency improvement from my perspective. However, of all of these, the separate tracking for each issue (rather than a single ginormous email) was the biggest improvement IMO.

In the end I felt that we were able to resolve all ~100 issues that were raised as part of AUTH48 completely and improve the document's accuracy and readability as a result. 

Here's what specifically worked well:
1) I wrote some scripting to parse the initial auth48 mail and turn that into ~60 separate github issues. This let each issue easily be discussed independently.
2) I triaged the issues, and those that seemed pretty straightforward I marked as 'editor-ready' and assigned to Jean, adding any necessary comments to the specific issue.
3) Those that weren't straightforward were assigned to members of the authoring team to track down the right solution, at which point they were then assigned as in 2).
4) When new questions came up, the editors or Jean would file new Github issues.
5) To address the issues assigned to her, Jean would create pull requests with her proposed changes, tagging the issues that the pull request was addressing.
6) I would review the pull request and either send comments back to Jean for further changes (through the pull request review tool) or merge the PR into the document.
7) When the PR was merged, the relevant issues were then closed, allowing us to easily track our progress via the size of the issues list.
8) In cases where the authoring team couldn't figure out a solution, we pulled relevant people (e.g., Adam Roach) into discussions to figure out a solution.

Here's what could have worked better:
a) as noted in 1) above, I had to write some Python to parse the email and call the Github API to create the issues. It would be better to have a tool that would let the RFC Editor directly create these issues, rather than trying to serialize to/from email.
b) we worked on a parallel branch to the main document, meaning that the main branch was frozen at jsep-24, and then a separate branch 'rfced' had the RFC Editor's version of the document, with all proposed changes and editor comments. This approach, while it avoided changing the 'master' branch with anything that wasn't fully vetted, ended up causing confusion. Working directly against master would have been easier.
c) the saga of #843 is a long and complicated story, but I think this issue is largely separate from the GitHub experiment. Neither the JSEP nor BUNDLE editors fully understood the conflict until it was too late, and a short meeting back in 2018 would probably have avoided the issue.

You are welcome to add my feedback to the experiment page - happy to provide more details as needed.

Justin

• The RPC needs to control the repository.
• The subject line of an issue needs to be succinct since the GUI truncates after 56 chars
• Each issue must contain just one topic.
• Ensure that the appropriate people are “Watch”ing the repo -- ADs, chairs, etc. (Only the owner of the repo can set people as collaborators. Only individuals can set their Watch settings for a repo.)

My biggest surprise - it is not as useful at communicating issues and changes as I thought it would be. One author was unaware of progress, and the AD never received notifications even when he was @mentioned. I also thought I knew a thing or two about Git and GitHub, but creating a workflow for the rtcweb-wg repo took time.

My sneaking suspicion - even frequent GitHub users are bitten by bad assumptions about how it works.

Things I liked - Being able to refer to issues and PRs with hashtags and have GitHub build hyperlinks in comments and commit messages. Automatic closure of issues when a PR was accepted is satisfying. Tracking of open issues is easy.

Things that were meh - interacting with Git and GitHub is not quick. Like other uses of GitHub I've seen in the IETF, technical discussions erupt on individual issues (see #843, which predated the AUTH48 issues but wasn't addressed by the authors until almost all the AUTH48 issues had been closed), and participants are slow to take the issue to a broader audience/ML (see the start of the email discussion).

Things that I would change next go around - the RPC should control the repo, which would allow us to merge the changes and add the right people as contributors so that they can (maybe) receive notifications (it may be a horse-to-water problem).