Standards Making Workflow

[chporter]

[chporter]

From the Governance document:
The Standards Working Group:

• oversees the adoption, dissemination, and administration of OMF standards for access, documentation, reusability, and interoperability;
• it may also propose additional standards.
• It proposes standards to be reviewed, revised, and approved by the OMF Members Council.
• The SWG works with the Executive Directorate and Administrative Coordinator to communicate OMF standards to OMF member organizations and other stakeholders, and solicit input from the modeling community.
• It also establishes procedures to evaluate and certify model code that meets OFM standards, working with code repositories and scientific software journals to ensure that code review includes OMF criteria and that OMF digital badges are awarded for models that comply with these standards.

[chporter]

Two functions of Standards Working Group:

1. Legislative - standards-making - designing, modifying,
2. Certification role - evaluate and certify model code

This discussion group focuses on issue 1.

[chporter]

• Procedure for review: loosely model on peer-review journal process.
• Standards will require different domain knowledge. Standards working group needs correct expertise. May need to pull in correct people from community to help with review.

1. Someone suggests an edit or creates an issue
2. Automated response upon submission of issue or PR - review process has been initiated.
3. "standards editor" decides level of review. (Who is the "standards editor"?)
   1. trivial (accept or reject as-is) (collect in one branch)
   2. minor - review by Standards Working Group only
   3. major (or new standard) - "public" review
4. For Minor modification:
   1. short timeline
   2. discussion in issues using "@user" for OMF community, and also pulling in experts as needed
   3. iterate on changes
   4. Implement in branch for each issue
5. For Major modification
   1. longer timeline
   2. solicit external review (experts in field), even outside OMF
   3. Discussion/ iteration of issues with SWG, experts
   4. Literature review (major revision). What is industry standard, if any?
   5. Testing phase (model this on the OGC workflow)
      • testbed for proposed standard
      • solicitation for review in community
   6. Iterate with OMF Members Council as needed
6. Periodic review of issues by OMF Members Council
   1. Working group makes recommendations to OMF MC for minor and major revisions
   2. OMF MC makes final decisions
   3. MC to resolve differences of opinion if they arise

[chporter]

• Versioning
  • Standards versioning. Stable snapshots of standards.
  • Models comply with a particular version of the standards.
• Need to establish a timeline of approval - deadline schedule for reviews
• For this workflow to be effective, we need to solidify standards and be very clear about what is required for compliance vs recommended for good practice.
• GitHub allows a review process. People are assigned. They can add comments.
• Iterative process of commits, reviews, etc.
• OMF provides a "seal of approval"
• Wikipedia model of modifications and review.
  • The more modifications you make, the more trusted (higher status) you are
  • May be wrong kind of incentive
  • Frequent standards-maker status (silver, gold, etc.)? use github tracking?
• Streamline on github site. Perhaps move to larger platform in the future as it's adopted.
• Standards Working group needs senior member oversight (veto power), at least at first.
• Members can volunteer to review standards. Can there be perks for this?
• Education: How can someone review and comment if they are not GitHub experts? This is especially important if we are asking outside experts to comment.
  • Form-based interface might help non-Github users.
  • Github training needed?
  • Model developers are probably OK with github.