repository: Add meta.json for repo metadata and cfsctl init#268
Draft
cgwalters wants to merge 6 commits intocomposefs:mainfrom
Draft
repository: Add meta.json for repo metadata and cfsctl init#268cgwalters wants to merge 6 commits intocomposefs:mainfrom
cgwalters wants to merge 6 commits intocomposefs:mainfrom
Conversation
crates/cfsctl/src/lib.rs
Outdated
| } | ||
|
|
||
| /// Handle `cfsctl init` | ||
| fn run_init(algorithm: &Algorithm, path: Option<&Path>, args: &App) -> Result<()> { |
Collaborator
There was a problem hiding this comment.
We can always create a meta.json for older repos as well, we should have all the info to do that. Right now we don't do that
Collaborator
Author
There was a problem hiding this comment.
Yes but that relates to #266 (comment)
IOW not having meta.json also means we can't read its splitstreams which is very useful information to know.
We could in theory though rewrite splitstreams from older versions realtively easily and then also create meta.json.
Collaborator
There was a problem hiding this comment.
Oh yeah. Had forgotten about that
Collaborator
Johan-Liebert1
left a comment
Johan-Liebert1
left a comment
There was a problem hiding this comment.
Just one comment, otherwise lgtm
jeckersb
reviewed
Mar 19, 2026
cgwalters
force-pushed
the
meta-json
branch
2 times, most recently
from
0df572b to
6ef4ab9
Compare
Collaborator
Author
|
OK there's now an even bigger change here, see the tip commit. I think it cleans up the UX a lot though. |
jeckersb
reviewed
Mar 20, 2026
crates/composefs/src/repository.rs
Outdated
| flock(&repository, FlockOperation::LockShared) | ||
| .context("Cannot lock composefs repository")?; | ||
|
|
||
| // Auto-detect security mode from meta.json's verity status. |
Collaborator
There was a problem hiding this comment.
This checks verity status on meta.json but doesn't read/parse/validate meta.json. Does it make sense to do that here so library users will get an early error if something is broken? Otherwise I think it's possible to open a repository and perform some subset of actions before it's later realized there's an issue.
auto-merge was automatically disabled
March 20, 2026 17:09
Pull request was converted to draft
Collaborator
Author
|
OK moving back to draft for some more refactoring |
cgwalters
force-pushed
the
meta-json
branch
2 times, most recently
from
8567826 to
7bd48fd
Compare
Add a validated Algorithm type that wraps the fsverity-<hash>-<lg_blocksize> string format (e.g. 'fsverity-sha512-12'). Implements FromStr for parsing with proper error types and Display for serialization, so it can be used as a clap value_parser argument. Includes for_hash::<H>() constructor to derive from FsVerityHashValue types at compile time. Prep for repository metadata support. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Prep for repository metadata (meta.json) serialization. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Add a meta.json file to the repository format that records the digest algorithm, format version, and feature flags, so tools can auto-detect the configuration instead of requiring --hash on every invocation. The versioning model is inspired by Linux filesystem superblocks (ext4, XFS, EROFS): a base version integer for fundamental layout changes, plus three tiers of feature flags for finer-grained evolution: - compatible: old tools can safely ignore - read-only-compatible: old tools may read but must not write - incompatible: old tools must refuse the repository entirely Because creating a repo is no longer just `mkdir`, add 'cfsctl init --algorithm=fsverity-sha512-12 [path]'. Closes: composefs#181 Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Prep for auto-detecting security mode from meta.json verity. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Derive the repository's insecure flag from whether meta.json has fs-verity enabled, rather than requiring a per-invocation --insecure flag. If meta.json carries verity, all objects are expected to have it too. We keep the `--insecure` CLI flag but it's a no-op as the semantics now changed to have fsverity enablement be controlled by the repo state. Via CLI and API one can override that to required - so we get a clean error if the repo doesn't have fsverity. Note that the composefs=? karg case uses this because source of truth has to be the karg, not the on-disk state. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
We need to handle the upgrade case in bootc, this is prep for that. When opening a repository without meta.json, check for the presence of objects/ to distinguish an old-format repo from an uninitialized directory. The error message directs users to `cfsctl init --reset-metadata` for migration. `init --reset-metadata` removes especially the splitstream data which changed format, but we keep the objects since those didn't change. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a meta.json file to the repository format that records the digest
algorithm, format version, and feature flags, so tools can auto-detect
the configuration instead of requiring --hash on every invocation.
The versioning model is inspired by Linux filesystem superblocks
(ext4, XFS, EROFS): a base version integer for fundamental layout
changes, plus three tiers of feature flags for finer-grained
evolution:
Because creating a repo is no longer just
mkdir, add'cfsctl init --algorithm=fsverity-sha512-12 [path]'.
Closes: #181