Yarn appears to be generating invalid SHA512 Checksums

[nigellh]

Apologies upfront, I am not a Yarn expert or user, but work with yarn.lock files to identify packages and dependencies.

Our team generate CDX SBOMs (Software Bill of Materials) that can be imported into Dependency Track (DT) which allows vulnerabilities to be identified.

The SBOMs are created from data found in Metadata files such as yarn.lock, package.json and many others.

A recent SBOM would not import into DT and it gets handed to me to find out why. It eventually came down to 538 invalid SHA512 Checksums that were in yarn.lock files.

An example entry in the SBOM is:

    {
      "type": "library",
      "bom-ref": "pkg:npm/[email protected]",
      "supplier": {
        "url": [
          "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz"
        ]
      },
      "author": "Raynos <[email protected]>",
      "name": "xtend",
      "version": "4.0.2",
      "description": "extend like a boss",
      "hashes": [
        {
          "alg": "SHA-512",
          "content": "10/ac5dfa738b21f6e7f0dd6e65e1b3155036d68104e67e5d5d1bde74892e327d7e5636a076f625599dc394330a731861e87343ff184b0047fef1360a7ec0a5a36a"
        }
      ],
      "licenses": [
        {
          "license": {
            "id": "MIT"
          }
        }
      ],
      "copyright": "Copyright (c) 2012-2014 Raynos.",
      "purl": "pkg:npm/[email protected]",
      "externalReferences": [
        {
          "type": "vcs",
          "url": "https://github.com/Raynos/xtend.git"
        }
      ],
      "properties": [
        {
          "name": "Relationship Completeness",
          "value": "Unknown"
        }
      ]
    },

The specific problem is

        {
          "alg": "SHA-512",
          "content": "10/ac5dfa738b21f6e7f0dd6e65e1b3155036d68104e67e5d5d1bde74892e327d7e5636a076f625599dc394330a731861e87343ff184b0047fef1360a7ec0a5a36a"
        }

According to the SBOM specification for hashes it can only contain:

"Must match regular expression: ^([a-fA-F0-9]{32}|[a-fA-F0-9]{40}|[a-fA-F0-9]{64}|[a-fA-F0-9]{96}|[a-fA-F0-9]{128})$"

So no /'s and can only be 128 characters long.

The problem SHA512 checksum is that it 131 characters long and has a / in.

In fact, every problem hash started 10/. I am guessing that whatever is creating these yarn.lock files is adding this "10/" to the front of them.

There is no hash in the package.json for this xtend package

We do not have access to the original software so cannot try to recreate it ourselves. This is what is in the yarn.lock file for the top of the yarn.lock file and for this package:

# This file is generated by running "yarn install" inside your project.
# Manual changes might be lost - proceed with caution!

__metadata:
  version: 8
  cacheKey: 10

"xtend@npm:^4.0.0, xtend@npm:~4.0.1":
  version: 4.0.2
  resolution: "xtend@npm:4.0.2"
  checksum: 10/ac5dfa738b21f6e7f0dd6e65e1b3155036d68104e67e5d5d1bde74892e327d7e5636a076f625599dc394330a731861e87343ff184b0047fef1360a7ec0a5a36a
  languageName: node
  linkType: hard

You can see by looking at this package's package.json file there is no hash in it, so we believe it must be the yarn command that created them.

Taking in the dev entries for the yarn.lock file, there are 1849 invalid hash entries.

Does anyone know if the yarn command creates this or have an idea where else I can look. Thanks in advance for any guidance.

(Cannot see anything in issues against this, but happy to raise one if this is thought to be a bug.)

[arcanis]

Whatever preceeds the / is the optional cache version. The checksum is only valid for archived generated by the same cache key (which may change when Yarn is updated, for example when we upgrade the libzip vendor).

[nigellh]

Any chance you can point to this optional cache version in some documentation? Not doubting you, but if we are going to code for it, I need to let the development team know what the spec for these are. Thanks.

[arcanis]

The lockfile format isn't part of the documented API, although we're mindful of downstream consumers whenever we update it.