[Feature]: Is there a way to remove useless metrics attributes?

[yunbo-xufeng]

Related Problems?

I'm using rust opentelemetry_sdk 0.19.0，and dump some metrics to opentelemetry collector，and one more things, the metrics attributes are changing always in my scenario, so the memory usage is increasing, and lead to oom issue finally.
So is there any way to cleanup these unused metrics attributes?

Describe the solution you'd like:

In my project, I also use Prometheus to dump metrics, to resolve the above issue, Prometheus provides a remove_label_values() api, so this is not a issue for Prometheus, could Opentelemetry SDK also provides a similar API?

Considered Alternatives

No response

Additional Context

No response

[cijothomas]

OTel SDK offers a View concept which can be leveraged to drop unwanted metric attributes.
https://github.com/open-telemetry/opentelemetry-rust/blob/main/examples/metrics-advanced/src/main.rs#L24-L31

Another alternative is, if you control the code, you can simply chose to add only the required metric attributes. If you cannot control the code, then View is the right solution from the SDK. (OTel Collector has similar mechanisms as well, if you are running OTel Collector in your environment)

[lalitb]

Curious, as we have the upper cardinality limit of 2000, the memory shouldn't be increasing continuously leading to OOM. What is the machine configuration? Eventually, we should be able to control the upper limit through Views (#1065). But as mentioned by @cijothomas, the right approach would be to filter the attributes through Views.

[cijothomas]

The cardinality capping is in the newer version, definitely not in 0.19.0!

[lalitb]

Ah, I overlooked the version

[yunbo-xufeng]

OTel SDK offers a View concept which can be leveraged to drop unwanted metric attributes. https://github.com/open-telemetry/opentelemetry-rust/blob/main/examples/metrics-advanced/src/main.rs#L24-L31

Another alternative is, if you control the code, you can simply chose to add only the required metric attributes. If you cannot control the code, then View is the right solution from the SDK. (OTel Collector has similar mechanisms as well, if you are running OTel Collector in your environment)

Thanks for your reply!
I just checked the View feature, and found it can only filter attribute keys, not including the values?
Actually, in my scenario, the metrics keys are not changed, only the values in the attributes changed when I call:

fn observe(&self, cx: &Context, value: T, attributes: &[KeyValue])

Any solution to resolve this "attributes key is constant but value is changing" problem?

[yunbo-xufeng]

Curious, as we have the upper cardinality limit of 2000, the memory shouldn't be increasing continuously leading to OOM. What is the machine configuration? Eventually, we should be able to control the upper limit through Views (#1065). But as mentioned by @cijothomas, the right approach would be to filter the attributes through Views.

Many thanks!
So the newer version OTel can resolve this memory problem implicitly by the cardinality limit?
My scenario is special: only the value changes, the key remains the same.

[cijothomas]

The new SDK will impose cardinality limit, so SDK wont go OOM.