Unit tests fail in Fedora 42 with distro's OpenSSL 3.2.4 #266
Comments
|
Hey @dagood, I was running into this failure when testing against our OpenSSL version 3.2.2. I reproduced the failure at 7d39d27 using the Dockerfiles from your post, other than adding
I noticed a few differences between the Debian and Fedora builds. The Fedora build is running the tests in FIPS mode with OpenSSL 3.2.4, while the Debian Bookworm build is running in non-FIPS mode with OpenSSL 3.0.15: Fedora (from your build.log): Debian (from my build): FWIU, FIPS 140-3 standards prohibit RSA with PKCS1v15 padding for encryption operations (though it's still allowed for signatures) because it's vulnerable to padding oracle attacks. OpenSSL 3.2.4 in FIPS mode correctly enforces this restriction.
The tests in the prior v2.0.0 release was passing because
Doesn't look like this is the case, as this failure is expected behavior for a FIPS-compliant implementation. What do you think about skipping these tests when FIPS mode is detected (unsure if this should be sensitive to the OpenSSL version)? And use RSA-OAEP padding instead for encryption operations in FIPS environments, since OAEP tests are already passing. The crypto/rsa PKCS1 encryption/decryption tests would probably need to be skipped in the crypto backend patches for https://github.com/microsoft/go as well. Please let me know if I am overlooking / misunderstanding something. I could contribute here as well. Thanks! :) |
|
Thanks for digging deeper!
Sorry, my wording wasn't clear--v2.0.0 also failed for me. That was simply the oldest version I attempted, not one that worked. 😄The failures are in
IMO, it's more justified to fix this now because:
@qmuntal would be the most affected by changes in this area and may have more context/opinions. 🙂 Skipping/avoiding this particular operation sounds right to me, but I don't have thoughts beyond that off the top of my head.
It's possible that this already works fine in microsoft/go (maybe upstream accounts for this FIPS behavior already or we already patched it), but it would be good to check. FWIW, golang-fips/openssl tests being more fragile wouldn't surprise me: they're (generally) only a developer resource while working on this library to exercise things, but the Go stdlib tests are more directly making sure that actual Go apps will work properly. |
|
Hey, thanks for your replies :)
I got around to investigating this. When running the tests in FIPS mode, a lot of tests fail for similar reasons, mainly RSA key sizes being too small. I created a best effort analysis with a table of test failures + outputs + causes. I'm not 100% certain on all of the causes, but mainly looks like test vectors that would need to be updated, or tests that can just be skipped. I've included the Dockerfile, scripts, and json output below: Dockerfile# ================================================
# == Fetch Go-FIPS Toolchain Build Dependencies ==
# ================================================
FROM fedora:42 AS bootstrap
# Install cgo-related dependencies
RUN set -eux; \
# Update all packages
dnf -y update; \
# Install required packages
dnf -y install \
ca-certificates \
crypto-policies \
crypto-policies-scripts \
gcc-c++ \
gcc \
git \
glibc-devel \
golang \
jq \
make \
openssl \
openssl-libs \
pkgconf \
sudo \
; \
# Clean up cache to reduce image size
dnf clean all
# For bootstrapping go build
ENV PATH="/root/go/bin:${PATH}"
RUN go install golang.org/dl/go1.24.1@latest \
&& go1.24.1 download
# Build Microsoft tooling
WORKDIR /go/src/msft-builder
RUN <<EOF
git clone --no-checkout https://github.com/microsoft/go .
git sparse-checkout init
git sparse-checkout set eng/_util
git checkout microsoft/release-branch.go1.24
cd eng/_util
go1.24.1 build -o /usr/local/bin/msft-builder ./cmd/build
EOF
# Fetch openssl shim for testing
WORKDIR /go/src/msft-openssl
RUN git clone --single-branch --branch v2 https://github.com/golang-fips/openssl .
# Download patched sources
WORKDIR /tmp
RUN <<EOF
set -e
curl -L -o go1.24.1.tar.gz \
https://github.com/microsoft/go/releases/download/v1.24.1-6/go1.24.1-20250312.2.src.tar.gz
mkdir -p /build
tar -xvf go1.24.1.tar.gz -C /build
EOF
# ==========================================
# == Verify OpenSSL/v2 Shim Compatibility ==
# ==========================================
FROM bootstrap AS smoketests
COPY --from=bootstrap /go/src/msft-openssl /go/src/msft-openssl
WORKDIR /go/src/msft-openssl
# Catch any openssl and FIPS mode compatibility errors before continuing
# Skipping PKCS1 encryption tests, see:
# https://github.com/golang-fips/openssl/issues/266
RUN CGO_ENABLED=1 go1.24.1 test \
-skip="TestRSAEncryptDecryptPKCS1_MissingPrecomputedValues|TestRSAEncryptDecryptPKCS1" -v ./...
# =============================================
# == Build the Go-FIPS Toolchain for Testing ==
# =============================================
FROM bootstrap AS buildgo
ARG GOEXPERIMENT
ARG GODEBUG
WORKDIR /build
ENV GOARCH=amd64
ENV GOOS=linux
ENV GOROOT_BOOTSTRAP=/root/sdk/go1.24.1
ENV GO_BUILDER_NAME="${GOOS}-${GOARCH}"
ENV CGO_ENABLED=1
RUN msft-builder -experiment opensslcrypto
# Configure OpenSSL to use FIPS provider
RUN cat > /etc/pki/tls/openssl_fips.cnf <<EOF
.include /etc/ssl/openssl.cnf
[openssl_init]
providers = provider_sect
alg_section = algorithm_sect
[provider_sect]
fips = fips_sect
[fips_sect]
activate = 1
[algorithm_sect]
default_properties = fips=yes
EOF
# Verify FIPS provider is available and loaded
ENV OPENSSL_CONF=/etc/pki/tls/openssl_fips.cnf
RUN openssl list -providers && openssl md5 /etc/hosts || echo "MD5 disabled as expected in FIPS mode"
# HACK: Replace Microsoft's env toggle with Fedora's env toggle
WORKDIR /build/go/src/cmd/go
RUN sed -i 's/OPENSSL_FORCE_FIPS_MODE/OPENSSL_CONF/g' script_test.go
# =====================================================
# == Get Failed Go-FIPS Toolchain Tests in FIPS mode ==
# =====================================================
FROM buildgo AS testfailures
WORKDIR /build
ENV GOEXPERIMENT=opensslcrypto,allowcryptofallback
ENV GODEBUG=fips140=on
# If this fails, that means that we will not be able to run tests in FIPS mode
RUN sudo --preserve-env /build/go/bin/go tool dist test \
-run "crypto/internal/backend/internal/opensslsetup.*"
# Get all failed tests
RUN sudo --preserve-env /build/go/bin/go tool dist test -json \
| jq -r 'select(.Action == "fail" and .Test != null)' > /tmp/failed.json
# Generate test regex string
COPY scripts/jq-filter-tests.sh scripts/jq-filter-tests.sh
RUN ./scripts/jq-filter-tests.sh | tee /tmp/skip-test-regex.txt
FROM scratch AS outputfailures
COPY --from=testfailures /tmp/failed-tests.json /
COPY --from=testfailures /tmp/skip-test-regex.txt /jq-filter-tests.sh#!/usr/bin/env bash
TMP_DIR="${TMP_DIR:-/tmp}"
# Filter extraneous fields and sort
jq -s 'map(del(.Time, .Elapsed)) | sort | group_by(.Package) | map({(.[0].Package): map(.Test)}) | add' "$TMP_DIR/failed.json" >"$TMP_DIR/failed-tests.json"
# Output generated test regex
TESTS=$(jq -r 'to_entries | map(select(.key)) | map(.key) | join("|")' "$TMP_DIR/failed-tests.json")
echo "!$TESTS"failed-tests.json{
"crypto/rsa": [
"TestDecryptPKCS1v15",
"TestEncryptDecryptOAEP",
"TestEncryptPKCS1v15",
"TestEncryptPKCS1v15DecrypterSessionKey",
"TestEncryptPKCS1v15SessionKey",
"TestEverything",
"TestEverything/1024",
"TestEverything/2048",
"TestKeyGeneration",
"TestKeyGeneration/512",
"TestPSS513",
"TestPSSGolden",
"TestPSSOpenSSL",
"TestPSSSigning",
"TestShortSessionKey",
"TestSignPKCS1v15",
"TestUnpaddedSignature",
"TestVerifyPKCS1v15"
],
"crypto/tls": [
"TestCrossVersionResume",
"TestCrossVersionResume/TLSv12",
"TestCrossVersionResume/TLSv13",
"TestFIPSCertAlgs",
"TestFIPSServerProtocolVersion",
"TestFIPSServerProtocolVersion/no-fips140tls",
"TestFIPSServerProtocolVersion/no-fips140tls/VersionTLS10",
"TestFIPSServerProtocolVersion/no-fips140tls/VersionTLS11",
"TestFIPSServerSignatureAndHash",
"TestFIPSServerSignatureAndHash/PKCS1WithSHA1",
"TestFIPSServerSignatureAndHash/PKCS1WithSHA1/no-fips140tls",
"TestKeysFromPreMasterSecret",
"TestSCTHandshake",
"TestSCTHandshake/TLSv12",
"TestSCTHandshake/TLSv13",
"TestTLS13OnlyClientHelloCipherSuite",
"TestTLS13OnlyClientHelloCipherSuite/empty",
"TestTLS13OnlyClientHelloCipherSuite/nil",
"TestTLS13OnlyClientHelloCipherSuite/some_TLS_1.2_and_1.3_ciphers",
"TestTLS13OnlyClientHelloCipherSuite/some_TLS_1.2_cipher",
"TestTLS13OnlyClientHelloCipherSuite/some_TLS_1.3_cipher",
"TestVersion"
],
"crypto/x509": [
"TestCRLCreation",
"TestCertificateOIDPoliciesGODEBUG",
"TestCertificatePolicies",
"TestCertificateRequestOverrides",
"TestCertificateRequestRoundtripFields",
"TestCreateCertificateRequest",
"TestCreateSelfSignedCertificate",
"TestDisableSHA1ForCertOnly",
"TestEmptySerialNumber",
"TestEmptySubject",
"TestMaxPathLen",
"TestMaxPathLenNotCA",
"TestNoAuthorityKeyIdInSelfSignedCert",
"TestNoSubjectKeyIdInCert",
"TestRejectCriticalAIA",
"TestRejectCriticalAKI",
"TestRejectCriticalSKI"
]
}Note: I did not use the powershell script but did use the Fedora FIPS Mode Test Failures@qmuntal Please let me know your thoughts. I can triage these failures further and help patch / skip these tests as well. :)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
My dev machine happens to be Fedora, and I noticed that the tests aren't working for me at ca56270. But... I checked all the tags since v2.0.0 and none of them succeeded with the current Fedora OpenSSL version, so this might just be some Fedora build quirk that might not be worth accounting for in the test suite.
To try to avoid env factors, I repro'd with a Dockerfile. Output from
podman build . > build.log:(The errors emitted also contain many trailing
nulcharacters that I had to remove from the file (build.log) to make copy-paste work. Maybe we aren't converting C string OpenSSL errors to Go strings properly? (Edit: fixed by #265.))The Dockerfile:
A similar Dockerfile based on
buildpack-deps:bookworm(debian) seems to work fine.Working Dockerfile
The text was updated successfully, but these errors were encountered: