From 71df9a6cbe5dde970b703e96e771abefb87e1de0 Mon Sep 17 00:00:00 2001 From: Adam Cmiel Date: Thu, 23 Jan 2025 17:02:33 +0100 Subject: [PATCH 1/3] prefetch: extract netrc docs into separate section The procedure for creating netrc secrets will be relevant for Go as well Signed-off-by: Adam Cmiel --- .../configuring/prefetching-dependencies.adoc | 83 ++++++++++--------- 1 file changed, 45 insertions(+), 38 deletions(-) diff --git a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc index 5f0e95c5..30c11646 100644 --- a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc +++ b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc @@ -216,7 +216,7 @@ spec: * From the {ProductName} *Applications* view, go to *Activity > Latest commits*. //include::../partials/con_hermetic_verification.adoc[] -=== Prefetching `pip` dependencies from custom index servers +=== [[custom-index-servers]]Prefetching `pip` dependencies from custom index servers Cachi2 supports the link:https://pip.pypa.io/en/stable/cli/pip_install/#install-index-url[--index-url] option. You can add this option to your `requirements.txt` file(s), instructing Cachi2 to download packages from the specified @@ -232,43 +232,8 @@ requests==2.32.2 \ # ...other packages ---- -WARNING: Do not include credentials in the index url. If needed, provide authentication through a `.netrc` file (as described below). - -To provide `.netrc` authentication for the index server: - -. Create a key/value secret (see xref:/how-tos/configuring/creating-secrets.adoc[creating secrets for your builds]). - Set `.netrc` as the key. Upload the content of your `.netrc` file as the value. For more details on netrc files, - review the link:https://pip.pypa.io/en/stable/topics/authentication/#netrc-support[pip documentation for netrc support]. - -. In your `.tekton/` PipelineRun files, add the newly created secret to the `.spec.workspaces` section: - -+ -[source,yaml] ----- -spec: - # ... - workspaces: - # ... - - name: netrc - secret: - secretName: my-netrc # the name you assigned to the secret in step 1 ----- - -. In the `.spec.pipelineSpec.tasks` section, find the entry with `name: prefetch-dependencies`. - Add the `netrc` workspace to the list of workspaces (if not present): - -+ -[source,yaml] ----- - tasks: - # ... - - name: prefetch-dependencies - # ... - workspaces: - # ... - - name: netrc - workspace: netrc ----- +WARNING: Do not include credentials in the index URL. If needed, provide authentication through a `.netrc` file (as described xref:netrc[below]). +For more pip-specific details on netrc files, review the link:https://pip.pypa.io/en/stable/topics/authentication/#netrc-support[pip documentation for netrc support]. == [[npm]]Enabling prefetch builds for `npm` Cachi2 supports `npm` by fetching any dependencies you declare in your `package.json` and `package-lock.json` project files. The npm CLI manages the `package-lock.json` file automatically, and Cachi2 fetches any dependencies and enables your build to install them without network access. @@ -499,6 +464,48 @@ In {ProductName}, from the *Applications* view, select the application build you * Select the *Logs* tab. * Alternatively, you can click *build-container*. When the right panel opens, select the *Logs* tab to see a partial view of the log for that build. +== [[netrc]]Creating the netrc secret + +The `prefetch-dependencies` task supports link:https://everything.curl.dev/usingcurl/netrc.html[.netrc] files for authentication. +For example, it is useful for: + +* xref:custom-index-servers[pip packages from private index servers] + +To create a netrc Secret for use in your pipelines: + +. Create the `.netrc` file locally. See the link:https://everything.curl.dev/usingcurl/netrc.html[.netrc] docs for more details on the file format. +. Create a key/value secret (see xref:/how-tos/configuring/creating-secrets.adoc[creating secrets for your builds]). + Set `.netrc` as the key. Upload the content of your `.netrc` file as the value. +. In your `.tekton/` PipelineRun files, add the newly created secret to the `.spec.workspaces` section: + ++ +[source,yaml] +---- +spec: + # ... + workspaces: + # ... + - name: netrc + secret: + secretName: my-netrc # the name you assigned to the secret in step 2 +---- + +. In the `.spec.pipelineSpec.tasks` section, find the entry with `name: prefetch-dependencies`. + Add the `netrc` workspace to the list of workspaces (if not present): + ++ +[source,yaml] +---- + tasks: + # ... + - name: prefetch-dependencies + # ... + workspaces: + # ... + - name: netrc + workspace: netrc +---- + == Additional resources * To troubleshoot any issues you might experience when you enable prefetch builds for `pip` or `pip` with source dependencies, see link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/pip.md#troubleshooting[Troubleshooting]. From 695a7599449567616f44155403c1fcf4e47809d7 Mon Sep 17 00:00:00 2001 From: Adam Cmiel Date: Thu, 23 Jan 2025 17:27:44 +0100 Subject: [PATCH 2/3] prefetch: document netrc for private Go modules Signed-off-by: Adam Cmiel --- .../how-tos/configuring/prefetching-dependencies.adoc | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc index 30c11646..1d656a6c 100644 --- a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc +++ b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc @@ -68,6 +68,12 @@ spec: * From the {ProductName} *Applications* view, go to *Activity > Latest commits*. //include::../partials/con_hermetic_verification.adoc[] +=== [[go-private]]Prefetching Go modules from private repos + +To enable downloading modules from private repos, Go supports authentication via `.netrc`. +You can create a netrc Secret for your pipeline as described xref:netrc[below]. +For more Go-specific `.netrc` details, see link:https://go.dev/doc/faq#git_https[the Go docs]. + == [[pip]]Enabling prefetch builds for `pip` Cachi2 supports pip by parsing of `pip` requirements files, including but not limited to, `requirements.txt` files placed in the root of your repository. By generically parsing `pip` requirements files, Cachi2 downloads the specified dependencies. @@ -470,6 +476,7 @@ The `prefetch-dependencies` task supports link:https://everything.curl.dev/using For example, it is useful for: * xref:custom-index-servers[pip packages from private index servers] +* xref:go-private[Go modules from private repos] To create a netrc Secret for use in your pipelines: From f0b7a604a46f6609be5e5c08e4c3a5fcf4bb8e82 Mon Sep 17 00:00:00 2001 From: Adam Cmiel Date: Fri, 24 Jan 2025 16:42:34 +0100 Subject: [PATCH 3/3] Add brief index of task input secrets The goal is to have a location listing notable secrets, but leave the documentation of those secrets in their more specialized documents. Signed-off-by: Adam Cmiel --- .../ROOT/pages/how-tos/configuring/creating-secrets.adoc | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/modules/ROOT/pages/how-tos/configuring/creating-secrets.adoc b/docs/modules/ROOT/pages/how-tos/configuring/creating-secrets.adoc index 9ef95c6c..5227277b 100644 --- a/docs/modules/ROOT/pages/how-tos/configuring/creating-secrets.adoc +++ b/docs/modules/ROOT/pages/how-tos/configuring/creating-secrets.adoc @@ -30,6 +30,13 @@ NOTE: One such task is the link:https://github.com/konflux-ci/build-definitions/ . Optional: Under **Labels**, add a label to tag or provide more context for your secret. . Click **Add secret**. +=== Notable task input secrets + +* xref:/how-tos/configuring/activation-keys-subscription.adoc#adding-activation-keys-to-the-workspace[activation-key] +* xref:/how-tos/configuring/prefetching-dependencies.adoc#creating-the-netrc-secret[netrc] +* xref:/how-tos/testing/build/snyk.adoc[snyk-secret] +* xref:/how-tos/testing/integration/third-parties/testing-farm.adoc[testing-farm-secret] + == Creating registry pull secrets Some container builds may use parent images from registries that require authentication, for example, `registry.redhat.io`. Until these credentials have been configured, the builds will continue to fail due to the system being unable to pull the required images.