NIST TestVectors for ML-KEM and ML-DSA

[iambatmanandjoker]

Hello,

Do we have any test vectors available from NIST already for ML-KEM and DSA.
i found some vectors attached here in the round 3 zip : https://pq-crystals.org/kyber/resources.shtml . Is it possible to test the same on liboqs ?

Regards.

[SWilson4]

Hi @iambatmanandjoker. We have tests for the NIST-provided ML-KEM and ML-DSA test vectors built into liboqs. You can run them by first building liboqs and then running

python3 -m pytest -v tests/test_vectors.py

from the main liboqs directory. Alternatively, if you run ninja run_tests from the build directory (further instructions in the liboqs README), these tests will be run as part of a larger test suite.

We don't currently check against intermediate test vectors for algorithms other than ML-KEM and ML-DSA. In particular, we don't test intermediate vectors for our "Round 3" Kyber and Dilithium implementations.

[iambatmanandjoker]

Hi @SWilson4 ,
I am able to run it locally successfully but via python it throws the below error. Any reason why tests are not supported on windows?

_platform win32 -- Python 3.12.2, pytest-8.1.1, pluggy-1.4.0 -- C:\Program Files\Python312\python.exe
cachedir: .pytest_cache
rootdir: C:\Work\base_liboqs\liboqs
plugins: xdist-3.5.0
collected 57 items

tests/test_vectors.py::test_vectors_kem[BIKE-L1] SKIPPED (Not needed on Windows) [ 1%]
tests/test_vectors.py::test_vectors_kem[BIKE-L3] SKIPPED (Not needed on Windows) [ 3%]
tests/test_vectors.py::test_vectors_kem[BIKE-L5] SKIPPED (Not needed on Windows) [ 5%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-348864] SKIPPED (Not needed on Windows) [ 7%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-348864f] SKIPPED (Not needed on Windows) [ 8%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-460896] SKIPPED (Not needed on Windows) [ 10%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-460896f] SKIPPED (Not needed on Windows) [ 12%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-6688128] SKIPPED (Not needed on Windows) [ 14%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-6688128f] SKIPPED (Not needed on Windows) [ 15%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-6960119] SKIPPED (Not needed on Windows) [ 17%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-6960119f] SKIPPED (Not needed on Windows) [ 19%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-8192128] SKIPPED (Not needed on Windows) [ 21%]
tests/test_vectors.py::test_vectors_kem[Classic-McEliece-8192128f] SKIPPED (Not needed on Windows) [ 22%]
tests/test_vectors.py::test_vectors_kem[HQC-128] SKIPPED (Not needed on Windows) [ 24%]
tests/test_vectors.py::test_vectors_kem[HQC-192] SKIPPED (Not needed on Windows) [ 26%]
tests/test_vectors.py::test_vectors_kem[HQC-256] SKIPPED (Not needed on Windows) [ 28%]
tests/test_vectors.py::test_vectors_kem[Kyber512] SKIPPED (Not needed on Windows) [ 29%]
tests/test_vectors.py::test_vectors_kem[Kyber768] SKIPPED (Not needed on Windows) [ 31%]
tests/test_vectors.py::test_vectors_kem[Kyber1024] SKIPPED (Not needed on Windows) [ 33%]
tests/test_vectors.py::test_vectors_kem[ML-KEM-512-ipd] SKIPPED (Not needed on Windows) [ 35%]
tests/test_vectors.py::test_vectors_kem[ML-KEM-512] SKIPPED (Not needed on Windows) [ 36%]
tests/test_vectors.py::test_vectors_kem[ML-KEM-768-ipd] SKIPPED (Not needed on Windows) [ 38%]
tests/test_vectors.py::test_vectors_kem[ML-KEM-768] SKIPPED (Not needed on Windows) [ 40%]
tests/test_vectors.py::test_vectors_kem[ML-KEM-1024-ipd] SKIPPED (Not needed on Windows) [ 42%]
tests/test_vectors.py::test_vectors_kem[ML-KEM-1024] SKIPPED (Not needed on Windows) [ 43%]
tests/test_vectors.py::test_vectors_kem[sntrup761] SKIPPED (Not needed on Windows) [ 45%]
tests/test_vectors.py::test_vectors_kem[FrodoKEM-640-AES] SKIPPED (Not needed on Windows) [ 47%]
tests/test_vectors.py::test_vectors_kem[FrodoKEM-640-SHAKE] SKIPPED (Not needed on Windows) [ 49%]
tests/test_vectors.py::test_vectors_kem[FrodoKEM-976-AES] SKIPPED (Not needed on Windows) [ 50%]
tests/test_vectors.py::test_vectors_kem[FrodoKEM-976-SHAKE] SKIPPED (Not needed on Windows) [ 52%]
tests/test_vectors.py::test_vectors_kem[FrodoKEM-1344-AES] SKIPPED (Not needed on Windows) [ 54%]
tests/test_vectors.py::test_vectors_kem[FrodoKEM-1344-SHAKE] SKIPPED (Not needed on Windows) [ 56%]
tests/test_vectors.py::test_vectors_sig[Dilithium2] SKIPPED (Not needed on Windows) [ 57%]
tests/test_vectors.py::test_vectors_sig[Dilithium3] SKIPPED (Not needed on Windows) [ 59%]
tests/test_vectors.py::test_vectors_sig[Dilithium5] SKIPPED (Not needed on Windows) [ 61%]
tests/test_vectors.py::test_vectors_sig[ML-DSA-44-ipd] SKIPPED (Not needed on Windows) [ 63%]
tests/test_vectors.py::test_vectors_sig[ML-DSA-44] SKIPPED (Not needed on Windows) [ 64%]
tests/test_vectors.py::test_vectors_sig[ML-DSA-65-ipd] SKIPPED (Not needed on Windows) [ 66%]
tests/test_vectors.py::test_vectors_sig[ML-DSA-65] SKIPPED (Not needed on Windows) [ 68%]
tests/test_vectors.py::test_vectors_sig[ML-DSA-87-ipd] SKIPPED (Not needed on Windows) [ 70%]
tests/test_vectors.py::test_vectors_sig[ML-DSA-87] SKIPPED (Not needed on Windows) [ 71%]
tests/test_vectors.py::test_vectors_sig[Falcon-512] SKIPPED (Not needed on Windows) [ 73%]
tests/test_vectors.py::test_vectors_sig[Falcon-1024] SKIPPED (Not needed on Windows) [ 75%]
tests/test_vectors.py::test_vectors_sig[Falcon-padded-512] SKIPPED (Not needed on Windows) [ 77%]
tests/test_vectors.py::test_vectors_sig[Falcon-padded-1024] SKIPPED (Not needed on Windows) [ 78%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHA2-128f-simple] SKIPPED (Not needed on Windows) [ 80%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHA2-128s-simple] SKIPPED (Not needed on Windows) [ 82%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHA2-192f-simple] SKIPPED (Not needed on Windows) [ 84%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHA2-192s-simple] SKIPPED (Not needed on Windows) [ 85%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHA2-256f-simple] SKIPPED (Not needed on Windows) [ 87%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHA2-256s-simple] SKIPPED (Not needed on Windows) [ 89%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHAKE-128f-simple] SKIPPED (Not needed on Windows) [ 91%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHAKE-128s-simple] SKIPPED (Not needed on Windows) [ 92%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHAKE-192f-simple] SKIPPED (Not needed on Windows) [ 94%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHAKE-192s-simple] SKIPPED (Not needed on Windows) [ 96%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHAKE-256f-simple] SKIPPED (Not needed on Windows) [ 98%]
tests/test_vectors.py::test_vectors_sig[SPHINCS+-SHAKE-256s-simple] SKIPPED (Not needed on Windows) [100%]_

[SWilson4]

I believe the reason is that those tests are written with UNIX-specific shell scripting. In general, our support for Windows (especially Win32) is not as complete as it is for Linux and macOS: see our platform support documentation for more details.

If you would find it useful to have these tests run on a particular platform (e.g., Win32), please feel free to submit a PR adding the functionality! We're always happy to receive contributions.

[iambatmanandjoker]

@SWilson4 also, can you please help answer the queries wrt Fifth PQC and ExpandMask question.
https://github.com/orgs/open-quantum-safe/discussions/1763
Thanks.

[SWilson4]

I'm not up to date on the specifics of ML-KEM and ML-DSA implementations. OQS pulls source code written by the the PQ-CRYSTALS team. They are the experts on that, and I would suggest asking your questions there.

[iambatmanandjoker]

Answered. will try to create a PR for Win32