Skip to content

Data race in dskit/ring #3893

Open
Open
@kolesnikovae

Description

@kolesnikovae

Discovered in CI tests in pkg/test/integration.

This has never been observed in the wild (we do not build with race, though). I'm not sure if the race is present in dskit/ring or in our tests.

==================
WARNING: DATA RACE
Write at 0x00c00d841860 by goroutine 40807:
  runtime.mapassign_faststr()
      /opt/hostedtoolcache/go/1.22.11/x64/src/runtime/map_faststr.go:203 +0x0
  github.com/grafana/dskit/ring.(*Desc).mergeWithTime()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/ring/model.go:222 +0xdda
  github.com/grafana/dskit/ring.(*Desc).Merge()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/ring/model.go:188 +0x5c
  github.com/grafana/dskit/kv/memberlist.computeNewValue()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:1426 +0x58
  github.com/grafana/dskit/kv/memberlist.(*KV).mergeValueForKey()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:1377 +0x2d3
  github.com/grafana/dskit/kv/memberlist.(*KV).mergeBytesValueForKey()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:1358 +0x1ad
  github.com/grafana/dskit/kv/memberlist.(*KV).MergeRemoteState()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:1319 +0x484
  github.com/hashicorp/memberlist.(*Memberlist).mergeRemoteState()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/net.go:1261 +0x25b
  github.com/hashicorp/memberlist.(*Memberlist).pushPullNode()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/state.go:664 +0x174
  github.com/hashicorp/memberlist.(*Memberlist).Join()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/memberlist.go:275 +0x4da
  github.com/grafana/dskit/kv/memberlist.(*KV).joinMembersBatch()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:678 +0xc4
  github.com/grafana/dskit/kv/memberlist.(*KV).joinMembersInBatches()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:655 +0x127
  github.com/grafana/dskit/kv/memberlist.(*KV).joinMembersWithRetries()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:600 +0x2c4
  github.com/grafana/dskit/kv/memberlist.(*KV).joinMembersOnStartup()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:575 +0x44b
  github.com/grafana/dskit/kv/memberlist.(*KV).running()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/kv/memberlist/memberlist_client.go:476 +0x6a
  github.com/grafana/dskit/kv/memberlist.(*KV).running-fm()
      <autogenerated>:1 +0x47
  github.com/grafana/dskit/services.(*BasicService).main()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/services/basic_service.go:190 +0x3b7
  github.com/grafana/dskit/services.(*BasicService).StartAsync.func1.gowrap1()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/services/basic_service.go:119 +0x33

Previous read at 0x00c00d841860 by goroutine 40943:
  runtime.mapiterinit()
      /opt/hostedtoolcache/go/1.22.11/x64/src/runtime/map.go:816 +0x0
  github.com/grafana/dskit/ring.(*Lifecycler).updateCounters()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/ring/lifecycler.go:920 +0x126
  github.com/grafana/dskit/ring.(*Lifecycler).initRing()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/ring/lifecycler.go:712 +0x7fc
  github.com/grafana/dskit/ring.(*Lifecycler).loop()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/ring/lifecycler.go:465 +0x68
  github.com/grafana/dskit/ring.(*Lifecycler).loop-fm()
      <autogenerated>:1 +0x47
  github.com/grafana/dskit/services.(*BasicService).main()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/services/basic_service.go:190 +0x3b7
  github.com/grafana/dskit/services.(*BasicService).StartAsync.func1.gowrap1()
      /home/runner/go/pkg/mod/github.com/grafana/[email protected]/services/basic_service.go:119 +0x33

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions