Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for verifying S/MIME messages #12267

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Add support for verifying S/MIME messages #12267

wants to merge 14 commits into from

Conversation

nitneuqr
Copy link
Contributor

As promised in #11555, I'm opening this PR with an initial implementation of S/MIME verification, in order to better discuss the API design, and to start the reviews while I finish some other features.

Namely, the new pkcs7_verify functions do not handle the certificate verification feature as of now. It as similar to a openssl smime -verify with the -noverify flag, to verify the signature but not the certificates (similar to what #12116 needs). Can you point me towards some existing code verifying X.509 certificates, if some exists?

Also, I have one question about the certificate parameter in the functions: should we verify against one certificates? Multiple ones? All the ones that are stored in the signature (if any)?

My essential thoughts for testing were to do the round-trip: signature using the PKCS7SignatureBuilder and verifying using the pkcs_decrypt functions. For now, I've not replaced the test_support.pkcs7_verify function, but I'm planning to do so as soon as the certificate verification feature is developed.

I'm still new to rust, so please let me know if you see some issues in variable lifetime, or some unnecessary copying between Python & Rust.

cc @alex

@nitneuqr nitneuqr changed the title Add support for decrypting S/MIME messages Add support for verifying S/MIME messages Jan 10, 2025
@alex
Copy link
Member

alex commented Jan 10, 2025 via email

nitneuqr
nitneuqr commented Jan 10, 2025
View reviewed changes
src/cryptography/hazmat/primitives/serialization/pkcs7.py Outdated
raise ValueError(
"Malformed multipart/signed message: must be multipart"
)
if not isinstance(payload[0], email.message.Message):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I'm not managing to create any test case (specific MIME message) that would raise this error. However, according to mypy, the case is still possible. Should I go with an assert in order to please mypy or have you got any idea on how the parts could be some text and not an email.message.Message?

@nitneuqr
Copy link
Contributor Author

@alex, for now I'm trying the following code:

def _verify_pkcs7_certificates(certificates: list[x509.Certificate]) -> None:
    builder = PolicyBuilder().store(Store(certificates))
    verifier = builder.build_client_verifier()
    verifier.verify(certificates[0], certificates[1:])

However, I'm getting the following error:

E       cryptography.hazmat.bindings._rust.x509.VerificationError: validation failed: basicConstraints.cA must not be asserted in an EE certificate (encountered processing <Certificate(subject=<Name(C=US,CN=cryptography CA)>, ...)>)

To be honest, I'm not clear as to what we are supposed to do in certificate verification. I've not found anything specific in the RFC, but I'll keep looking. If you can guide me through this, I'd love some help 🛩️

@alex
Copy link
Member

alex commented Jan 13, 2025

Basically certificates used as end entities (i.e., the cert used to sign a PKCS#7/SMIME message) should not hvae ca=true in their basic constraints extension. I can't remember which spec that comes from, but its a general best practice in any event /cc @woodruffw

@woodruffw
Copy link
Contributor

Yep, that comes from CABF -- we have a test and cite for it here: https://x509-limbo.com/testcases/webpki/#webpkica-as-leaf.

(RFC 5280 doesn't impose the requirement that the EE not be a CA, but firmly agreed about it being best practice.)

@nitneuqr
Copy link
Contributor Author

Basically certificates used as end entities (i.e., the cert used to sign a PKCS#7/SMIME message) should not hvae ca=true in their basic constraints extension. I can't remember which spec that comes from, but its a general best practice in any event /cc @woodruffw

OK, that raises two things for me:

  • That means we will have to create a specific certificate for verifying (and signing ?) PKCS#7 files, either statically or dynamically. I'll look into it.
  • This raises the question of test_support.pkcs7_verify function. It does not check this, as we can see in test_support.rs:93, by just passing an empty stack of certificates.
let certs = openssl::stack::Stack::new()?;

p7.verify(
    &certs,
    &store,
    msg.as_ref().map(|m| m.as_bytes()),
    None,
    flags,
)?;

@nitneuqr
Copy link
Contributor Author

@alex this should be ready for review, the tests are passing. A few considerations to keep in mind:

  • What should the priority be for user-input parameters vs. what's found in the PKCS7 structure (content, certificate)?
  • Handling PKCS#7 message with multiple signers: trying to verify the first signer? All of them?
  • Missing the documentation for the new pkcs#7 test vectors; still WIP.

Keep in mind that for now, I've used a certificate chain from endesive to make the tests pass, but I'll work on creating new cryptography certificates in the following days.

Some more things about the certs:

  • the certificate used for testing PKCS#7 signing cannot be verified properly (due to some EE error above)
  • Verifying the signed messages, when using openssl, only works when some other certificates are passed in the -CAfile parameter. Otherwise we are getting a unable to get local issuer certificate error. This is not necessary (for some reason that I do not understand) in the new functions.

CC @reaperhulk @facutuesca if you miraculously have time for a review 😄

@nitneuqr
Copy link
Contributor Author

Stumbled on #12104 and realized we might have the same issue with the certificate verification code!

@nitneuqr nitneuqr requested a review from alex January 20, 2025 15:12
@alex
Copy link
Member

alex commented Jan 20, 2025 via email

@nitneuqr
Copy link
Contributor Author

No worries! Just checking in in case you forgot about it 😊

@prauscher
Copy link
Contributor

Stumbled on #12104 and realized we might have the same issue with the certificate verification code!

Not exactly sure, but first huge thanks for the work: After reading your PR, your Usecase seems to be "I want to verify a received SMIME-Message" (along what I paraphased in #12104 as verify_message-method of a Verifier to be obtained using build_smime_verifier). My usecase is more "I want to verify a Certificate to use it for sending a SMIME-Encrypted message" (described as verify_certificate).

However, my Usecase should be a subtask of your Usecase: To verify a received SMIME-Message you probably need to verify the used Certificate. Looks like you are doing this in _verify_pkcs7_certificate, which is using build_client_verifier - however from my knowledge, this requires EKU_CLIENT_AUTH_OID to be set, but SMIME-Certificates can be fine without client auth (and require EKU_EMAIL_PROTECTION_OID instead.

So in my (really basic) knowledge, your current code would give invalid validation results for Certificates which are only used for SMIME-Signing (and not client authentication). However, once you include a verifier for it, please make it publicly accessible to use it without a signed Message :)

@alex alex mentioned this pull request Jan 28, 2025
Closed
@alex
Copy link
Member

alex commented Jan 31, 2025

Looking at your questions:

What should the priority be for user-input parameters vs. what's found in the PKCS7 structure (content, certificate)?

What if we start with the most restrictive implementation: it is an error for both to be present?

Handling PKCS#7 message with multiple signers: trying to verify the first signer? All of them?

I think for an MVP we should start with an "any" policy: if any of the signers matches what we want, then it's ok. (In the future maybe there's a use case for an "all" policy? I'm not sure)

alex
alex reviewed Jan 31, 2025
View reviewed changes
src/rust/src/pkcs7.rs Outdated
Comment on lines 868 to 873
let certificates = pyo3::types::PyList::empty(py);
certificates.append(certificate)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to add an argument for people to provide a trust store

@nitneuqr
Copy link
Contributor Author

nitneuqr commented Feb 2, 2025

Waiting for #12360 before going any further on this :)

@lgcCerti lgcCerti mentioned this pull request Feb 6, 2025
Closed
@lgcCerti
Copy link

lgcCerti commented Feb 6, 2025

Hello, I opened #12406 wondering about this implementation.
Would a PKCS7 verification be compatible with CMS? Such as the CMS_verify used in openssl-cms ?

According to OpenSSL they are very similar, but I couldn't pinpoint the edge cases

@alex
Copy link
Member

alex commented Feb 14, 2025

FYI, #12360 is merged now.

@nitneuqr
PKCS7 signing (first version)
bdfd759 
no certificate verification as of now

handling PEM, DER, SMIME formats

added tests & documentation accordingly
@nitneuqr
doing assertions for now, to please mypy
3908af0
@nitneuqr
added more test coverage
c2287aa
@nitneuqr
updated tests to avoid unsupported algorithm
d70307f
@nitneuqr
first failing code for certificate verification
edc74c1
@nitneuqr
handling mixed types with Cow
8eed98f
@nitneuqr
feat: functions now have optional keyword arguments
12bc3e8 
certificate is now optional

feat: handling RSA case

feat: No signature parameter

adapted tests accordingly
@nitneuqr
Copy link
Contributor Author

nitneuqr commented Feb 15, 2025 •
edited
Loading

FYI, #12360 is merged now.

Will try to work on this the following weeks. I just read https://github.com/nitneuqr/cryptography/blob/pkcs7-verify/docs/x509/verification.rst#L284; as far as I understand, it seems we need to create a dedicated S/MIME extension policy and verify based on this policy?

Do you know where you can find information on the required information for S/MIME certificates?

Update: just re-read all the comments and it seems at least part of the info is there 😄

@nitneuqr
Copy link
Contributor Author

nitneuqr commented Feb 15, 2025 •
edited
Loading

As far as I understand it now, it seems that we'll need first to create a S/MIME extension policy (probably in another PR), test it with some certificates, and use the same certificates to test S/MIME verification. Does that make sense @alex?

Update: opened #12465 for that matter.

@nitneuqr nitneuqr mentioned this pull request Feb 15, 2025
Open
@alex
Copy link
Member

alex commented Feb 15, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alex alex alex left review comments

@Tarnum-tst Tarnum-tst Tarnum-tst approved these changes

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@nitneuqr @alex @woodruffw @prauscher @lgcCerti @Tarnum-tst