Skip to content

Validation: unify CLI reports and add a strict publish profile#951

Draft
zkwentz wants to merge 1 commit into
rfc008-09-author-validation-rfcfrom
rfc008-10-publish-profile
Draft

Validation: unify CLI reports and add a strict publish profile#951
zkwentz wants to merge 1 commit into
rfc008-09-author-validation-rfcfrom
rfc008-10-publish-profile

Conversation

@zkwentz

@zkwentz zkwentz commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

What & Why

Publishing needs a stricter answer than ordinary local iteration: every blocking criterion must actually run and pass. This PR unifies CLI execution on the shared planner and adds the strict publish profile.

Risk & Review Guidance

  • Risk level: MEDIUM — changes validation exit semantics for the new publish path
  • Task class: feature
  • Review focus: src/openenv/cli/commands/validate.py:30 for unified dispatch; src/openenv/validation/models.py:626 for publish aggregation; tests/test_validation/test_executor.py for blocking-skip behavior
  • Blast radius: Incorrect aggregation could publish an incomplete environment or block a valid one.

Decisions & Alternatives

  • Blocking skip fails only the publish gate — static/runtime/full remain useful for partial local iteration.
  • Remove the legacy semantic split — rejected maintaining a second validator with different criteria.
  • Remote defaults to publish — isolation is primarily the pre-publish author path.

Verification

  • PASS: PYTHONPATH=src:envs .venv/bin/python -m pytest tests/ -q -m 'not integration' — 1,559 passed, 114 skipped, 31 external integration tests deselected on the complete stack.
  • PASS: Fork whole-stack CI — 7/7 checks passed, including Python 3.11/3.12, lint, docs, lock validation, and package smoke tests.
  • NOT verified: No live remote execution is part of this layer.

Scope & Non-Goals

  • This PR does not yet create the remote Sandbox or wire the gate into openenv push.

Breaking Changes / Migration Notes

The new publish profile intentionally exits non-zero for blocking skips; existing static/runtime/full behavior remains less strict.

Stack

PR 10 of 17; depends on #950.

Full 17-PR stack
  1. #942 RFC 008: Spec-neutral local and remote validation architecture
  2. #943 Validation foundation: spec-neutral contracts and adapters
  3. #944 Validation policy: spec-aware check registry and planner
  4. #945 Validation executor: hardened shared reports and eligibility
  5. #946 Local validation API: execution-model-aware profiles
  6. #947 CLI: validation profiles and shared reports
  7. #948 HF runtime: dedicated sandbox execution mode
  8. #949 HF certification: enforce trusted dedicated isolation
  9. #950 [RFC 008] Define remote author validation and publish gating
  10. #951 Validation: unify CLI reports and add a strict publish profile ← current
  11. #952 Validation: add structured author diagnostics and remediation
  12. #953 Validation: run author validation in dedicated HF Sandboxes
  13. #954 Validation: isolate submitted code in author Sandboxes
  14. #955 Validation: bind author reports to policy and source
  15. #956 CLI: scaffold publish requirements
  16. #957 CLI: gate pushes on staged remote validation
  17. #958 Docs: explain local, remote, and publish validation

All upstream PRs remain draft while RFC 008 is under review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant