golang docker image based CI jobs may not be using the expected go version #36080
Description
The golang docker image sets GOTOOLCHAIN=local which means unless something else sets GOTOOLCHAIN env, go will prefer the version in the docker image. The default GOTOOLCHAIN value in a local go install is GOTOOLCHAIN=auto (go env GOTOOLCHAIN will show this)
Setting it to local makes sense if you want to tightly control the go version from the environment by selecting the docker image version.
In most of our projects, I think we expect that with GOTOOLCHAIN support, CI will behave like local development, and use the version of go selected by the project (through go.mod or other means) so they don't have to update CI config constantly, and having CI use the same version is just to avoid having to download go again at runtime.
GOTOOLCHAIN=auto means you can test updating go in a PR to the repo under test.
For our own images, I sent updates:
- set GOTOOLCHAIN=auto so automatic verison selection can be used release#4215
- set gotoolchain=auto #36074
There may be other images that require this update, and there are jobs that use the golang images directly.
For jobs that use golang directly, and use >= go 1.21, we should consider setting GOTOOLCHAIN=auto
I think could technically set it as a ci-wide default to be overridden by the jobs, the same way we set GOPROXY:
test-infra/config/prow/config.yaml
Lines 1126 to 1129 in 0b316ac
This uses a preset that targets all jobs by not having a specific selector.
That feature may be a little surprising though, and I'm not actually sure the order of defaulting if a job attempts to override it in CI config on the pod spec versus within the job's scripts.