cmd/go: spurious "v1.x.y is not a tag" error when a tag's commit was previously download without the tag

[central182]

What version of Go are you using (go version)?

$ go version
go version go1.18.4 linux/amd64

Does this issue reproduce with the latest release?

Yes. I was able to reproduce from Docker image golang:1.18.4.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY="github.com/central182"
GONOSUMDB="github.com/central182"
GOOS="linux"
GOPATH="/go"
GOPRIVATE="github.com/central182"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.4"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/root/example/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build856782336=/tmp/go-build -gno-record-gcc-switches"

What did you do?

On my host machine

1. Create a private Go module. (Say it's hosted on https://github.com/central182/foobar and the latest commit is ef2f664, with tag v1.0.9 pointing to some earlier commit.)
2. Launch a Docker container that is able to access my private repositories on GitHub.

$ docker run -it -v /home/me/shared_ssh:/root/.ssh golang:1.18.4

Inside Docker

3. Do the necessary configuration for downloading private Go modules via SSH.

$ export GOPRIVATE=github.com/central182
$ git config --global url."ssh://[email protected]".insteadOf "https://github.com"

4. Create a temporary Go module.

$ cd ~ && mkdir example && cd example && go mod init example
go: creating new go.mod: module example

5. Get the foobar module with its latest commit hash ef2f664.

$ go get github.com/central182/foobar@ef2f664
go: downloading github.com/central182/foobar v1.0.10-0.20220226164108-ef2f6641951f
go get: added github.com/central182/foobar v1.0.10-0.20220226164108-ef2f6641951f

On my host machine

6. Tag ef2f664 with v1.0.10 and push it to GitHub.

$ git tag v1.0.10
$ git push --tags
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:central182/foobar.git
 * [new tag]         v1.0.10 -> v1.0.10
$ git log --oneline
ef2f664 (HEAD -> main, tag: v1.0.10, origin/main) Empty commit

Inside Docker

7. Try to get the foobar module with the tag v1.0.10, and watch it fail.

$ go get github.com/central182/[email protected]
go get: github.com/central182/[email protected]: invalid version: resolves to version v1.0.11-0.20220226164108-ef2f6641951f (v1.0.10 is not a tag)

8. However, if I repeat step 7 again, the download will succeed.

$ go get github.com/central182/[email protected]
go: downloading github.com/central182/foobar v1.0.10
go get: upgraded github.com/central182/foobar v1.0.10-0.20220226164108-ef2f6641951f => v1.0.10

What did you expect to see?

No failure at step 7.

What did you see instead?

The tag did exist while go said otherwise.

[central182]

I have posted this issue before as #51374 but was told it was duplicate to #51312.
Now that the fix to #51312 is released with 1.18.4 (according to #53588) I believe they are different issues.
Possibly related to #51312 (comment)?

[bcmills]

Thanks for the detailed repro steps — I had not seen the prior issue you filed. I agree that this looks like a different root cause.

[bcmills]

This is plausibly related to #56881.

[markbastiaans]

Also hitting this issue with Go 1.20.4, private modules (GOPRIVATE set, git insteadOf configured) on an Azure Pipelines runner container (based on Ubuntu 22.04).

I cannot reproduce this issue locally, so my suspicion is that that this issue might be related to the containerized environment.

[bcmills]

I have a test script that reproduces this behavior locally; I hope to have a fix ready for Go 1.22 or shortly thereafter.

[gopherbot]

Change https://go.dev/cl/547155 mentions this issue: cmd/go/internal/modfetch/codehost: add new git tags before statLocal instead of after

[bcmills]

@gopherbot, please backport to Go 1.21 and 1.20. This fixes a bug that is otherwise difficult for users to understand and diagnose, and the fix is relatively uninvasive.

[gopherbot]

Backport issue(s) opened: #64585 (for 1.20), #64586 (for 1.21).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[gopherbot]

Change https://go.dev/cl/548215 mentions this issue: cmd/go: set global .gitconfig location more robustly in tests

[bcmills]

If we backport this fix, it needs to include both CL 547155 and CL 548215 to avoid breaking the builders.

[gopherbot]

Change https://go.dev/cl/585815 mentions this issue: [release-branch.go1.21] cmd/go/internal/modfetch/codehost: add new git tags before statLocal instead of after

[gopherbot]

Change https://go.dev/cl/585816 mentions this issue: [release-branch.go1.21] cmd/go: set global .gitconfig location more robustly in tests