Skip to content

Exponential histogram observations mismatch #6006

@sfc-gh-dguy

Description

@sfc-gh-dguy

Describe the bug
We upgraded to Micrometer 1.14.0 and started using exponential histograms around 2 months ago.
Since day 1 of using exponential histograms we noticed this error on our metrics backend (Grafana GEM):

ts=2025-03-05T05:07:37.899310792Z caller=grpc_logging.go:76 level=warn method=/cortex.Ingester/Push duration=3.033186ms msg=gRPC err="user=default-tenant: err: 8 observations found in buckets, but the Count field is 10: histogram's observation count should equal the number of observations found in the buckets (in absence of NaN). timestamp=2025-03-05T05:07:36.402Z, series=[{__name__ proxy_convergence_duration}] (err-mimir-native-histogram-count-mismatch) (sampled 1/10)

It's saying total count of observations in buckets doen't match the Count field. This seems like a bug

  • Micrometer version: 1.14.0
  • Micrometer registry: OTLP
  • OS: linux
  • Java version: 11

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions