Skip to content

feat(encryption) [8/N] Read encrypted manifest file#2586

Open
xanderbailey wants to merge 3 commits into
apache:mainfrom
xanderbailey:xb/encrypted_manifest_file_read
Open

feat(encryption) [8/N] Read encrypted manifest file#2586
xanderbailey wants to merge 3 commits into
apache:mainfrom
xanderbailey:xb/encrypted_manifest_file_read

Conversation

@xanderbailey

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

Read decrypted manifest files. This one is pretty straightforward, if the ManifestFile has key metadata on it, we should use that to construct an EncryptedInputFile and read the file using that.

Are these changes tested?

Roundtrip test added showing that we can write and then read encrypted manifest files.

@xanderbailey xanderbailey changed the title Read encrypted manifest file feat(encryption) [8/N] Read encrypted manifest file Jun 4, 2026

@mbutrovich mbutrovich left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is correct and minimal - decode StandardKeyMetadata from self.key_metadata and read through EncryptedInputFile when present, plaintext otherwise - and it's nice that the roundtrip test exercises the AAD prefix.

Just a test request.

let input = file_io.new_input(&self.manifest_path)?;
let avro = match &self.key_metadata {
Some(key_metadata_bytes) => {
let key_metadata = StandardKeyMetadata::decode(key_metadata_bytes)?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same hardcoded-decode point as #2584 (so I won't re-litigate it here) - flagging only that if the FileKeyResolver lands, this path needs to be included, and there's a threading wrinkle: load_manifest(&self, file_io: &FileIO) has no parameter to receive a resolver today. Whoever lands the seam will need to thread it (or an EncryptionManager) into load_manifest.

}

#[tokio::test]
async fn test_load_manifest_decrypts_when_key_metadata_present() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The roundtrip test is good. For parity with the data-file tests in #2584 (which cover missing-key and wrong-key), consider a wrong-key (or wrong-AAD) case asserting load_manifest fails cleanly, so a future change that silently mis-resolves the key is caught.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a couple of tests

@xanderbailey xanderbailey force-pushed the xb/encrypted_manifest_file_read branch from d415799 to 4c7e2df Compare July 1, 2026 10:50
Add failure-case coverage for load_manifest alongside the existing
roundtrip test: decrypting with the wrong DEK or the wrong AAD prefix
must fail cleanly rather than silently returning garbage. Extract a
shared write_encrypted_manifest helper to avoid duplicating the writer
setup across the three tests.
@xanderbailey

Copy link
Copy Markdown
Contributor Author

Thanks for the review @mbutrovich. Have added additional tests. @blackmwk would you be able to take a look here when you have a moment?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants