Start Publishing Staging Images to Artifact Registry #3961

upodroid · 2022-07-12T16:08:40Z

Part of: #1343

Notes: https://gist.github.com/upodroid/a33723a7e1abc5e9c6fabc6b07e7aac0

When images are built after a PR is merged, they need to be pushed to Artifact Registry(AR).

Unlike Google Container Registry(GCR), you don't need a separate GCP project per staging project. The permission boundary is at the repository level instead of the project. This allows multiple registries to be created per project. Therefore, we will create a new project, start publishing images to it and delete the other projects after the transition.

We will need to do the following:
- create a new GCP project (k8s-artifacts-staging)
- create a docker image repository per project
- modify the jobs to push images to prow.
- Provision the infra with terraform. There is a separate issue open to deploy changes to terraform via Prow on merges.

Prod changes need to be done via the shell scripts which makes deploying staging via terraform kind of pointless.

Open questions:

Do we need to backfill the staging repository via gcrane?

@puerco

/area artifacts
/priority important-soon
/area release-eng

upodroid · 2022-07-13T11:34:58Z

After #3968 is merged, we need to do a few things:

repos that use ko to build images(k-sigs/bom,etc), modify KO_DOCKER_REPO variable to us-docker.pkg.dev/k8s-staging-bom/images/bom as an example
Modify image-builder to tweak the cloudbuild.yaml file

Bit unsure about other things.

upodroid · 2022-07-26T17:46:47Z

Spoke about this at the sig-testing meeting. Will push it out to the future when GCR deprecation is actually announced by Google.

May want to standardize on Aritfact Registry eventually, but would require tracking down a number of staging registries and changing team push processes (with unclear payoff right now)

/priority backlog
/sig testing

ameukam · 2022-08-25T19:44:53Z

/milestone v1.26

k8s-triage-robot · 2022-11-23T20:03:01Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

ameukam · 2022-12-21T10:49:40Z

/remove-lifecycle stale
/milestone v1.27
/priority important-longterm

k8s-triage-robot · 2023-03-21T11:20:06Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2023-04-20T11:35:37Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

dims · 2024-06-17T20:28:21Z

Need to go back and create etcd-manager staging bucket in #6897

slack conversation : https://kubernetes.slack.com/archives/CCK68P2Q2/p1718654806739639

k8s-triage-robot · 2024-09-22T20:07:22Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2024-10-22T20:42:46Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

ameukam · 2024-10-23T01:27:22Z

/remove-lifecycle rotten

ameukam · 2024-12-18T00:52:36Z

/milestone v1.33

BenTheElder · 2024-12-18T01:06:53Z

(Some projects are on AR, but not all, thanks to @upodroid)

ameukam · 2025-02-12T21:37:24Z

We have exceptions for migration:

https://github.com/kubernetes/k8s.io/blob/main/infra/gcp/terraform/k8s-staging-images/registries.tf#L35-L41

upodroid · 2025-03-11T17:19:45Z

Quick update, I'm on track to finish the ACTIVE projects by the deadline by Google(18th of March)

Outstanding projects:

# some of these have been moved to k8s-staging-images AR instances such as #7824
 mahamed  Mahameds-MacBook-Pro  ~  1  USAGE  $  gcloud container images list-gcr-usage     --folder=713040427754 --filter="usage=ACTIVE"
---
repository: gcr.io/k8s-staging-lws
usage: ACTIVE
---
repository: gcr.io/k8s-staging-jobset
usage: ACTIVE
---
repository: gcr.io/k8s-staging-provider-os
usage: ACTIVE
---
repository: gcr.io/k8s-staging-tejolote
usage: ACTIVE
---
repository: gcr.io/k8s-staging-zeitgeist
usage: ACTIVE
---
repository: gcr.io/k8s-staging-kmm
usage: ACTIVE
---
repository: gcr.io/k8s-staging-kwok
usage: ACTIVE
---
repository: gcr.io/k8s-staging-win-op-rdnss
usage: ACTIVE
---
repository: gcr.io/k8s-staging-kueue
usage: ACTIVE
---
repository: gcr.io/k8s-staging-gmsa-webhook
usage: ACTIVE
---
repository: gcr.io/k8s-staging-sched-simulator
usage: ACTIVE
---
repository: gcr.io/k8s-staging-prometheus-adapter
usage: ACTIVE
---
repository: gcr.io/k8s-staging-gateway-api
usage: ACTIVE
---
repository: gcr.io/k8s-staging-releng-test
usage: ACTIVE
---
repository: gcr.io/k8s-staging-kustomize
usage: ACTIVE
---
repository: gcr.io/k8s-staging-networking
usage: ACTIVE
---
repository: gcr.io/k8s-staging-sp-operator
usage: ACTIVE
---
repository: gcr.io/k8s-staging-sig-docs
usage: ACTIVE
---
repository: gcr.io/k8s-staging-scheduler-plugins
usage: ACTIVE
---
repository: gcr.io/k8s-staging-git-sync
usage: ACTIVE
---
repository: gcr.io/k8s-staging-sig-storage
usage: ACTIVE
---
repository: gcr.io/k8s-staging-ingressconformance
usage: ACTIVE
---
repository: gcr.io/k8s-staging-ingress-nginx
usage: ACTIVE
---
repository: gcr.io/k8s-staging-provider-aws
usage: ACTIVE
---
repository: gcr.io/k8s-staging-kube-state-metrics
usage: ACTIVE
---
repository: gcr.io/k8s-staging-slack-infra
usage: ACTIVE
---
repository: gcr.io/k8s-staging-kubernetes
usage: ACTIVE
---
repository: gcr.io/k8s-staging-scl-image-builder
usage: ACTIVE
---
repository: gcr.io/k8s-staging-nfd
usage: ACTIVE
---
repository: gcr.io/k8s-staging-npd
usage: ACTIVE
---
repository: gcr.io/k8s-staging-external-dns
usage: ACTIVE
---
repository: gcr.io/k8s-staging-provider-azure
usage: ACTIVE
---
repository: gcr.io/k8s-staging-kas-network-proxy
usage: ACTIVE
---
repository: gcr.io/k8s-staging-multitenancy
usage: ACTIVE
---
repository: gcr.io/k8s-staging-publishing-bot
usage: ACTIVE

Successfully Migrated Projects Projects:

 mahamed  Mahameds-MacBook-Pro  ~  1  $  gcloud container images list-gcr-usage     --folder=713040427754 --filter="usage=REDIRECTED"
---
repository: gcr.io/k8s-staging-cloud-pv-vsphere
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-ipam-ic
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cloud-pv-labeler
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-api-helm
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-operator
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-cloudstack
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-bom
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cloud-provider-ibm
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-ibmcloud
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-api-nested
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-test-infra
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cri-tools
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-ci-images
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-addon-manager
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-addons
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cloud-provider-gcp
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-boskos
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-infra-tools
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cpa
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-etcd
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-csi-secrets-store
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-dns
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-api-do
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-autoscaling
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-vsphere
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-apisnoop
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-e2e-test-images
usage: REDIRECTED
---
repository: asia.gcr.io/k8s-staging-cip-test
usage: REDIRECTED
---
repository: eu.gcr.io/k8s-staging-cip-test
usage: REDIRECTED
---
repository: us.gcr.io/k8s-staging-cip-test
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-api-azure
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-api-gcp
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-docker
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-openstack
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-artifact-promoter
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-capi-kubeadm
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-descheduler
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-build-image
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-api-aws
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cip-test
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-coredns
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-csi
usage: REDIRECTED
---
repository: gcr.io/k8s-staging-cluster-api
usage: REDIRECTED

Projects with inactive GCR usage. They will be migrated in late March after the ACTIVE projects are done.

 mahamed  Mahameds-MacBook-Pro  ~  1  $  gcloud container images list-gcr-usage     --folder=713040427754 --filter="usage=INACTIVE"
---
repository: gcr.io/k8s-staging-wasm-scheduler
usage: INACTIVE
---
repository: gcr.io/k8s-staging-kubebuilder
usage: INACTIVE
---
repository: gcr.io/k8s-staging-win-svc-proxy
usage: INACTIVE
---
repository: gcr.io/k8s-staging-tg-exporter
usage: INACTIVE
---
repository: gcr.io/k8s-staging-krm-functions
usage: INACTIVE
---
repository: gcr.io/k8s-staging-sig-auth
usage: INACTIVE
---
repository: gcr.io/k8s-staging-kubetest2
usage: INACTIVE
---
repository: gcr.io/k8s-staging-provider-openstack
usage: INACTIVE
---
repository: gcr.io/k8s-staging-experimental
usage: INACTIVE
---
repository: gcr.io/k8s-staging-examples
usage: INACTIVE
---
repository: gcr.io/k8s-staging-storage-migrator
usage: INACTIVE
---
repository: gcr.io/k8s-staging-etcdadm
usage: INACTIVE
---
repository: gcr.io/k8s-staging-k8s-gsm-tools
usage: INACTIVE
---
repository: gcr.io/k8s-staging-kubeadm
usage: INACTIVE

upodroid added the sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. label Jul 12, 2022

upodroid mentioned this issue Jul 13, 2022

Deploy Artifact Registry to Production Project #3968

Merged

upodroid mentioned this issue Jul 13, 2022

Google Container Registry to Artifact Registry Migration Timelines #3976

Closed

k8s-ci-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Jul 26, 2022

riaankleinhans moved this to Implementation TODO in registry.k8s.io (SIG K8S Infra) Jul 26, 2022

riaankleinhans added this to registry.k8s.io (SIG K8S Infra) Jul 26, 2022

ameukam removed the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Aug 1, 2022

k8s-ci-robot added this to the v1.26 milestone Aug 25, 2022

riaankleinhans moved this from Implementation TODO to Breakdown TODO in registry.k8s.io (SIG K8S Infra) Sep 15, 2022

ameukam removed this from registry.k8s.io (SIG K8S Infra) Oct 7, 2022

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 23, 2022

k8s-ci-robot added priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 21, 2022

k8s-ci-robot modified the milestones: v1.26, v1.27 Dec 21, 2022

riaankleinhans added this to registry.k8s.io (SIG K8S Infra) Jan 4, 2023

riaankleinhans moved this to Backlog in registry.k8s.io (SIG K8S Infra) Jan 4, 2023

riaankleinhans moved this from Backlog to artifact.k8s.io in registry.k8s.io (SIG K8S Infra) Jan 8, 2023

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 21, 2023

dims mentioned this issue Jun 17, 2024

Setup image release for etcd-manager #6897

Merged

dims removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jun 17, 2024

dims assigned upodroid Jun 17, 2024

This was referenced Jun 18, 2024

Deploy new k8s-staging-images project #6905

Merged

migrate etcd-manager to AR #6906

Merged

k8s-ci-robot closed this as completed in #6905 Jun 18, 2024

github-project-automation bot moved this to Reporting, Audit & Logging in registry.k8s.io (SIG K8S Infra) Jun 18, 2024

ameukam reopened this Jun 24, 2024

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 22, 2024

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Oct 22, 2024

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Oct 23, 2024

k8s-ci-robot added this to the v1.33 milestone Dec 18, 2024

upodroid mentioned this issue Feb 3, 2025

Create new infra-tools AR registry #7750

Merged

upodroid pinned this issue Feb 3, 2025

upodroid mentioned this issue Feb 24, 2025

create jobset and lws image registries #7824

Merged

thockin mentioned this issue Mar 13, 2025

gcr container registry not online anymore soon kubernetes/git-sync#940

Closed

upodroid mentioned this issue Mar 22, 2025

create AR registries with lifecycle policies #7913

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Start Publishing Staging Images to Artifact Registry #3961

Start Publishing Staging Images to Artifact Registry #3961

upodroid commented Jul 12, 2022 •

edited

Loading

upodroid commented Jul 13, 2022

upodroid commented Jul 26, 2022

ameukam commented Aug 25, 2022

k8s-triage-robot commented Nov 23, 2022

ameukam commented Dec 21, 2022

k8s-triage-robot commented Mar 21, 2023

k8s-triage-robot commented Apr 20, 2023

dims commented Jun 17, 2024

k8s-triage-robot commented Sep 22, 2024

k8s-triage-robot commented Oct 22, 2024

ameukam commented Oct 23, 2024

ameukam commented Dec 18, 2024

BenTheElder commented Dec 18, 2024

ameukam commented Feb 12, 2025

upodroid commented Mar 11, 2025

Start Publishing Staging Images to Artifact Registry #3961

Start Publishing Staging Images to Artifact Registry #3961

Comments

upodroid commented Jul 12, 2022 • edited Loading

upodroid commented Jul 13, 2022

upodroid commented Jul 26, 2022

ameukam commented Aug 25, 2022

k8s-triage-robot commented Nov 23, 2022

ameukam commented Dec 21, 2022

k8s-triage-robot commented Mar 21, 2023

k8s-triage-robot commented Apr 20, 2023

dims commented Jun 17, 2024

k8s-triage-robot commented Sep 22, 2024

k8s-triage-robot commented Oct 22, 2024

ameukam commented Oct 23, 2024

ameukam commented Dec 18, 2024

BenTheElder commented Dec 18, 2024

ameukam commented Feb 12, 2025

upodroid commented Mar 11, 2025

upodroid commented Jul 12, 2022 •

edited

Loading