Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support sha512 content #543

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Support sha512 content #543

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 0 additions & 26 deletions .github/workflows/conformance-action-pr.yml
View file
Open in desktop

This file was deleted.

7 changes: 6 additions & 1 deletion .github/workflows/conformance-action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
push:
branches:
- main
pull_request:
branches:
- main

jobs:
run:
Expand All @@ -12,12 +15,14 @@ jobs:
- name: checkout source code
uses: actions/checkout@v3
- name: Start a test registry (zot)
id: registry-zot
run: |
set -x
make registry-ci
echo "port=$(docker port oci-conformance 5000 | head -1 | cut -f2 -d:)" >>$GITHUB_OUTPUT
- name: Run OCI distribution-spec conformance
env:
OCI_ROOT_URL: http://localhost:5000
OCI_ROOT_URL: http://localhost:${{ steps.registry-zot.outputs.port }}
OCI_NAMESPACE: myorg/myrepo
OCI_TEST_PULL: 1
OCI_TEST_PUSH: 1
Expand Down
32 changes: 19 additions & 13 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,27 +92,33 @@ conformance-test:

conformance-binary: $(OUTPUT_DIRNAME)/conformance.test

TEST_REGISTRY_CONTAINER ?= ghcr.io/project-zot/zot-minimal-linux-amd64:v2.0.0-rc6@sha256:bf95a94849cd9c6f596fb10e5a2d03b74267e7886d1ba0b3dab33337d9e46e5c
# TODO: update image once changes are merged in zot
# TEST_REGISTRY_CONTAINER ?= ghcr.io/project-zot/zot-minimal-linux-amd64:v2.1.0
TEST_REGISTRY_CONTAINER ?= ghcr.io/andaaron/zot-minimal-linux-amd64:v2.1.0-manifest-digest
registry-ci:
docker rm -f oci-conformance && \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having all commands joined by && ensures make executes them in the same shell.

In the new code on the right side they seem to be executed in separate shells since they are different rows in the make target. (I also don't see .ONESHELL being used). I don't think this is intended, even if it is unlikely someone would use make -j

It may be worth it to double check the new behavior of the make target is correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my limited testing, make -j ... would still run all steps within a single job in order. Removing the && was intended to let make better report failures when one step in the middle of a long list of steps fails. Without that, users are guessing if it's the mkdir, or which of the two docker commands output some error message. However, I'm far from a make expert, so if there are errors this creates, send a reproducible since I'd like to better understand this.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind, all is ok, please resolve

mkdir -p $(OUTPUT_DIRNAME) && \
echo '{"distSpecVersion":"1.1.0-dev","storage":{"rootDirectory":"/tmp/zot","gc":false,"dedupe":false},"http":{"address":"0.0.0.0","port":"5000"},"log":{"level":"debug"}}' > $(shell pwd)/$(OUTPUT_DIRNAME)/zot-config.json
docker run -d \
-v $(shell pwd)/$(OUTPUT_DIRNAME)/zot-config.json:/etc/zot/config.json \
--name=oci-conformance \
-p 5000:5000 \
$(TEST_REGISTRY_CONTAINER) && \
sleep 5

conformance-ci:
export OCI_ROOT_URL="http://localhost:5000" && \
docker rm -f oci-conformance || true
mkdir -p $(OUTPUT_DIRNAME)
docker run -d --rm \
--name=oci-conformance \
-p 5000 \
$(TEST_REGISTRY_CONTAINER)
sleep 5

conformance-ci: conformance-binary
export OCI_ROOT_URL="http://localhost:$$(docker port oci-conformance 5000 | head -1 | cut -f2 -d:)" && \
export OCI_NAMESPACE="myorg/myrepo" && \
export OCI_TEST_PULL=1 && \
export OCI_TEST_PUSH=1 && \
export OCI_TEST_CONTENT_DISCOVERY=1 && \
export OCI_TEST_CONTENT_MANAGEMENT=1 && \
$(shell pwd)/$(OUTPUT_DIRNAME)/conformance.test

conformance-clean:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my other comment on this file, same applies here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't apply here, these commands are independent. let's mark this as resolved.

docker stop oci-conformance || true
[ ! -f $(OUTPUT_DIRNAME)/conformance.test ] || rm "$(OUTPUT_DIRNAME)/conformance.test"
[ ! -f "junit.xml" ] || rm junit.xml
[ ! -f "report.html" ] || rm report.html

$(OUTPUT_DIRNAME)/conformance.test:
cd conformance && \
CGO_ENABLED=0 go test -c -o $(shell pwd)/$(OUTPUT_DIRNAME)/conformance.test \
Expand Down
2 changes: 1 addition & 1 deletion action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ runs:
run: |
set -x

# Enter the directory containing the checkout of this action which is surpisingly hard to do (but we did it... #OCI)
# Enter the directory containing the checkout of this action which is surprisingly hard to do (but we did it... #OCI)
cd "$(dirname $(find $(find ~/work/_actions -name distribution-spec -print -quit) -name Makefile -print -quit))"

# The .git folder is not present, but the dirname is the requested action ref, so use this as the conformance version
Expand Down
163 changes: 149 additions & 14 deletions conformance/01_pull_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ package conformance

import (
"net/http"
"os"

"github.com/bloodorangeio/reggie"
g "github.com/onsi/ginkgo/v2"
Expand All @@ -12,8 +11,6 @@ import (
var test01Pull = func() {
g.Context(titlePull, func() {

var tag string

g.Context("Setup", func() {
g.Specify("Populate registry with test blob", func() {
SkipIfDisabled(pull)
Expand Down Expand Up @@ -72,9 +69,8 @@ var test01Pull = func() {
g.Specify("Populate registry with test manifest", func() {
SkipIfDisabled(pull)
RunOnlyIf(runPullSetup)
tag = testTagName
req := client.NewRequest(reggie.PUT, "/v2/<name>/manifests/<reference>",
reggie.WithReference(tag)).
reggie.WithReference(testTagName)).
SetHeader("Content-Type", "application/vnd.oci.image.manifest.v1+json").
SetBody(manifests[0].Content)
resp, err := client.Do(req)
Expand All @@ -98,14 +94,41 @@ var test01Pull = func() {
BeNumerically("<", 300)))
})

g.Specify("Get tag name from environment", func() {
g.Specify("Populate registry with sha512 blobs", func() {
SkipIfDisabled(pull)
RunOnlyIfNot(runPullSetup)
tmp := os.Getenv(envVarTagName)
if tmp != "" {
tag = tmp
RunOnlyIf(runPull512Setup)
for _, blob := range testBlobs["sha512"] {
req := client.NewRequest(reggie.POST, "/v2/<name>/blobs/uploads/").
SetQueryParam("digest-algorithm", "sha512")
resp, err := client.Do(req)
Expect(err).To(BeNil())
req = client.NewRequest(reggie.PUT, resp.GetRelativeLocation()).
SetQueryParam("digest", blob.Digest).
SetHeader("Content-Type", "application/octet-stream").
SetHeader("Content-Length", blob.ContentLength).
SetBody(blob.Content)
resp, err = client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(SatisfyAll(
BeNumerically(">=", 200),
BeNumerically("<", 300)))
}
})

g.Specify("Populate registry with test sha512 manifest", func() {
SkipIfDisabled(pull)
RunOnlyIf(runPull512Setup)
req := client.NewRequest(reggie.PUT, "/v2/<name>/manifests/<reference>",
reggie.WithReference(testTag512Name)).
SetQueryParam("digest", testManifests["sha512"].Digest).
SetHeader("Content-Type", "application/vnd.oci.image.manifest.v1+json").
SetBody(testManifests["sha512"].Content)
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(SatisfyAll(
BeNumerically(">=", 200),
BeNumerically("<", 300)))
})
})

g.Context("Pull blobs", func() {
Expand All @@ -130,6 +153,18 @@ var test01Pull = func() {
}
})

g.Specify("HEAD request to existing sha512 blob should yield 200", func() {
SkipIfDisabled(pull)
req := client.NewRequest(reggie.HEAD, "/v2/<name>/blobs/<digest>",
reggie.WithDigest(testBlobs["sha512"][0].Digest))
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
if h := resp.Header().Get("Docker-Content-Digest"); h != "" {
Expect(h).To(Equal(testBlobs["sha512"][0].Digest))
}
})

g.Specify("GET nonexistent blob should result in 404 response", func() {
SkipIfDisabled(pull)
req := client.NewRequest(reggie.GET, "/v2/<name>/blobs/<digest>",
Expand All @@ -146,6 +181,15 @@ var test01Pull = func() {
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
})

g.Specify("GET request to existing sha512 blob URL should yield 200", func() {
SkipIfDisabled(pull)
req := client.NewRequest(reggie.GET, "/v2/<name>/blobs/<digest>",
reggie.WithDigest(testBlobs["sha512"][0].Digest))
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
})
})

g.Context("Pull manifests", func() {
Expand Down Expand Up @@ -182,10 +226,23 @@ var test01Pull = func() {
}
})

g.Specify("HEAD request to sha512 manifest (digest) should yield 200 response", func() {
SkipIfDisabled(pull)
req := client.NewRequest(reggie.HEAD, "/v2/<name>/manifests/<digest>",
reggie.WithDigest(testManifests["sha512"].Digest)).
SetHeader("Accept", "application/vnd.oci.image.manifest.v1+json")
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
if h := resp.Header().Get("Docker-Content-Digest"); h != "" {
Expect(h).To(Equal(testManifests["sha512"].Digest))
}
})

g.Specify("HEAD request to manifest path (tag) should yield 200 response", func() {
SkipIfDisabled(pull)
Expect(tag).ToNot(BeEmpty())
req := client.NewRequest(reggie.HEAD, "/v2/<name>/manifests/<reference>", reggie.WithReference(tag)).
Expect(testTagName).ToNot(BeEmpty())
req := client.NewRequest(reggie.HEAD, "/v2/<name>/manifests/<reference>", reggie.WithReference(testTagName)).
SetHeader("Accept", "application/vnd.oci.image.manifest.v1+json")
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expand All @@ -195,6 +252,19 @@ var test01Pull = func() {
}
})

g.Specify("HEAD request to sha512 manifest (tag) should yield 200 response", func() {
SkipIfDisabled(pull)
Expect(testTag512Name).ToNot(BeEmpty())
req := client.NewRequest(reggie.HEAD, "/v2/<name>/manifests/<reference>", reggie.WithReference(testTag512Name)).
SetHeader("Accept", "application/vnd.oci.image.manifest.v1+json")
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
if h := resp.Header().Get("Docker-Content-Digest"); h != "" {
Expect(h).To(Equal(testManifests["sha512"].Digest))
}
})

g.Specify("GET nonexistent manifest should return 404", func() {
SkipIfDisabled(pull)
req := client.NewRequest(reggie.GET, "/v2/<name>/manifests/<reference>",
Expand Down Expand Up @@ -222,10 +292,29 @@ var test01Pull = func() {
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
})

g.Specify("GET request to sha512 manifest (digest) should yield 200 response", func() {
SkipIfDisabled(pull)
req := client.NewRequest(reggie.GET, "/v2/<name>/manifests/<digest>", reggie.WithDigest(testManifests["sha512"].Digest)).
SetHeader("Accept", "application/vnd.oci.image.manifest.v1+json")
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
})

g.Specify("GET request to manifest path (tag) should yield 200 response", func() {
SkipIfDisabled(pull)
Expect(tag).ToNot(BeEmpty())
req := client.NewRequest(reggie.GET, "/v2/<name>/manifests/<reference>", reggie.WithReference(tag)).
Expect(testTagName).ToNot(BeEmpty())
req := client.NewRequest(reggie.GET, "/v2/<name>/manifests/<reference>", reggie.WithReference(testTagName)).
SetHeader("Accept", "application/vnd.oci.image.manifest.v1+json")
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(Equal(http.StatusOK))
})

g.Specify("GET request to sha512 manifest (tag) should yield 200 response", func() {
SkipIfDisabled(pull)
Expect(testTag512Name).ToNot(BeEmpty())
req := client.NewRequest(reggie.GET, "/v2/<name>/manifests/<reference>", reggie.WithReference(testTag512Name)).
SetHeader("Accept", "application/vnd.oci.image.manifest.v1+json")
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expand Down Expand Up @@ -285,6 +374,20 @@ var test01Pull = func() {
Equal(http.StatusMethodNotAllowed),
))
})
g.Specify("Delete sha512 manifest created in setup", func() {
SkipIfDisabled(pull)
RunOnlyIf(runPull512Setup)
req := client.NewRequest(reggie.DELETE, "/v2/<name>/manifests/<digest>", reggie.WithDigest(testManifests["sha512"].Digest))
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(SatisfyAny(
SatisfyAll(
BeNumerically(">=", 200),
BeNumerically("<", 300),
),
Equal(http.StatusMethodNotAllowed),
))
})
}

g.Specify("Delete config[0] blob created in setup", func() {
Expand Down Expand Up @@ -331,6 +434,24 @@ var test01Pull = func() {
))
})

for _, blob := range testBlobs["sha512"] {
g.Specify("Delete blob created in setup", func() {
SkipIfDisabled(pull)
RunOnlyIf(runPull512Setup)
req := client.NewRequest(reggie.DELETE, "/v2/<name>/blobs/<digest>", reggie.WithDigest(blob.Digest))
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(SatisfyAny(
SatisfyAll(
BeNumerically(">=", 200),
BeNumerically("<", 300),
),
Equal(http.StatusNotFound),
Equal(http.StatusMethodNotAllowed),
))
})
}

if !deleteManifestBeforeBlobs {
g.Specify("Delete manifest[0] created in setup", func() {
SkipIfDisabled(pull)
Expand Down Expand Up @@ -360,6 +481,20 @@ var test01Pull = func() {
Equal(http.StatusMethodNotAllowed),
))
})
g.Specify("Delete sha512 manifest created in setup", func() {
SkipIfDisabled(pull)
RunOnlyIf(runPull512Setup)
req := client.NewRequest(reggie.DELETE, "/v2/<name>/manifests/<digest>", reggie.WithDigest(testManifests["sha512"].Digest))
resp, err := client.Do(req)
Expect(err).To(BeNil())
Expect(resp.StatusCode()).To(SatisfyAny(
SatisfyAll(
BeNumerically(">=", 200),
BeNumerically("<", 300),
),
Equal(http.StatusMethodNotAllowed),
))
})
}
})
})
Expand Down
Loading
Loading