RSE Wiki Archive

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Feedback from 2020-2021 Experiment: Using GitHub for AUTH48 =====

Please see [[github_auth48_experiment]] for context.

==== Feedback from Justin Uberti (author, RFC 8829) ====

<code>
Summary:
This was the first time I've had such a long document go through the 
AUTH48 process, but from my perspective things worked much better than 
they would have using email.

There were literally almost 100 separate changes to the text during AUTH48 
(not sure if this is typical or not for this size of document). Being able 
to track each of these individually, set the appropriate status for each 
issue, distribute the work across the authoring team and Jean, bring people 
in to comment on specific issues, and finally review proposed changes with 
full diffs and ability to easily discuss the diffs - each of these was a 
significant efficiency improvement from my perspective. However, of all 
of these, the separate tracking for each issue (rather than a single 
ginormous email) was the biggest improvement IMO.

In the end I felt that we were able to resolve all ~100 issues that were 
raised as part of AUTH48 completely and improve the document's accuracy 
and readability as a result. 

Here's what specifically worked well:
1) I wrote some scripting to parse the initial auth48 mail and turn that 
into ~60 separate github issues. This let each issue easily be discussed 
independently.
2) I triaged the issues, and those that seemed pretty straightforward 
I marked as 'editor-ready' and assigned to Jean, adding any necessary 
comments to the specific issue.
3) Those that weren't straightforward were assigned to members of the 
authoring team to track down the right solution, at which point they 
were then assigned as in 2).
4) When new questions came up, the editors or Jean would file new Github issues.
5) To address the issues assigned to her, Jean would create pull 
requests with her proposed changes, tagging the issues that the pull 
request was addressing.
6) I would review the pull request and either send comments back to 
Jean for further changes (through the pull request review tool) or 
merge the PR into the document.
7) When the PR was merged, the relevant issues were then closed, 
allowing us to easily track our progress via the size of the issues list.
8) In cases where the authoring team couldn't figure out a solution, 
we pulled relevant people (e.g., Adam Roach) into discussions to figure 
out a solution.

Here's what could have worked better:
a) as noted in 1) above, I had to write some Python to parse the email and call 
the Github API to create the issues. It would be better to have a tool that would 
let the RFC Editor directly create these issues, rather than trying to serialize 
to/from email.
b) we worked on a parallel branch to the main document, meaning that the main 
branch was frozen at jsep-24, and then a separate branch 'rfced' had the 
RFC Editor's version of the document, with all proposed changes and editor 
comments. This approach, while it avoided changing the 'master' branch with 
anything that wasn't fully vetted, ended up causing confusion. Working directly 
against master would have been easier.
c) the saga of #843 is a long and complicated story, but I think this issue is 
largely separate from the GitHub experiment. Neither the JSEP nor BUNDLE editors 
fully understood the conflict until it was too late, and a short meeting back 
in 2018 would probably have avoided the issue.

You are welcome to add my feedback to the experiment page - happy to provide 
more details as needed.

Justin
</code>


--------------------------------------- 
==== Feedback from RFC Editor (regarding RFC 8829) ====

=== Lessons Learned ===

  * The RPC needs to control the repository.
  * The subject line of an issue needs to be succinct since the GUI truncates after 56 chars
  * Each issue must contain just one topic.
  * Ensure that the appropriate people are "Watch"ing the repo -- ADs, chairs, etc. (Only the owner of the repo can set people as collaborators. Only individuals can set their Watch settings for a repo.)  

=== High-level thoughts about this round with GitHub ===


My biggest surprise - it is not as useful at communicating issues and changes as I thought it would be. One author was unaware of progress, and the AD never received notifications even when he was @mentioned.  I also thought I knew a thing or two about Git and GitHub, but creating a workflow for the rtcweb-wg repo took time.

My sneaking suspicion - even frequent GitHub users are bitten by bad assumptions about how it works.

Things I liked - Being able to refer to issues and PRs with hashtags and have GitHub build hyperlinks in comments and commit messages. Automatic closure of issues when a PR was accepted is satisfying. Tracking of open issues is easy.

Things that were meh - interacting with Git and GitHub is not quick. Like other uses of GitHub I've seen in the IETF, technical discussions erupt on individual issues (see [[https://github.com/rtcweb-wg/jsep/issues/843|#843]], which predated the AUTH48 issues but wasn't addressed by the authors until almost all the AUTH48 issues had been closed), and participants are slow to take the issue to a broader audience/ML (see [[https://www.rfc-editor.org/pipermail/c238/2021-January/001306.html|the start of the email discussion]]).

Things that I would change next go around - the RPC should control the repo, which would allow us to merge the changes and add the right people as contributors so that they can (maybe) receive notifications (it may be a horse-to-water problem).
RSE Wiki Archive

User Tools

Site Tools

Page Tools
