Skip to content

Commit 60abbf4

Browse files
SylvainJugejaydeluca
SylvainJuge
and
jaydeluca
authored
jmx add jvm metrics yaml (#13392)
Co-authored-by: Jay DeLuca <[email protected]>
1 parent 0fef809 commit 60abbf4

File tree

3 files changed

+362
-0
lines changed

3 files changed

+362
-0
lines changed

‎instrumentation/jmx-metrics/library/jvm.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# JVM Metrics
2+
3+
Here is the list of metrics based on MBeans exposed by the JVM and that are defined in [`jvm.yaml`](./src/main/resources/jmx/rules/jvm.yaml).
4+
5+
Those metrics are defined in the [JVM runtime metrics semantic conventions](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/).
6+
7+
| Metric Name | semconv maturity | Type | Attributes | Description |
8+
|---------------------------------------------------------------------------------------------------------------------------------------|:-----------------|---------------|---------------------------------------|----------------------------------------------------|
9+
| [jvm.memory.used](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryused) | stable | UpDownCounter | jvm.memory.pool.name, jvm.memory.type | Used memory |
10+
| [jvm.memory.committed](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemorycommitted) | stable | UpDownCounter | jvm.memory.pool.name, jvm.memory.type | Committed memory |
11+
| [jvm.memory.limit](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemorylimit) | stable | UpDownCounter | jvm.memory.pool.name, jvm.memory.type | Max obtainable memory |
12+
| [jvm.memory.init](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryinit) | experimental | UpDownCounter | jvm.memory.pool.name, jvm.memory.type | Initial memory requested |
13+
| [jvm.memory.used_after_last_gc](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryused_after_last_gc) | stable | UpDownCounter | jvm.memory.pool.name, jvm.memory.type | Memory used after latest GC |
14+
| [jvm.thread.count](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmthreadcount) | stable | UpDownCounter | [^1] | Threads count |
15+
| [jvm.class.loaded](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmclassloaded) | stable | Counter | | Classes loaded since JVM start |
16+
| [jvm.class.unloaded](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmclassunloaded) | stable | Counter | | Classes unloaded since JVM start |
17+
| [jvm.class.count](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmclasscount) | stable | UpDownCounter | | Classes currently loaded count |
18+
| [jvm.cpu.count](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmcpucount) | stable | UpDownCounter | | Number of CPUs available |
19+
| [jvm.cpu.recent_utilization](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmcpurecent_utilization) | stable | Gauge | | Recent CPU utilization for process reported by JVM |
20+
| [jvm.system.cpu.load_1m](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmsystemcpuload_1m) | experimental | Gauge | | Average CPU load reported by JVM |
21+
| [jvm.system.cpu.recent_utilization](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmcpurecent_utilization) | experimental | Gauge | | Recent CPU utilization reported by JVM |
22+
| [jvm.buffer.memory.used](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmbuffermemoryused) | experimental | UpDownCounter | jvm.buffer.pool.name | Memory used by buffers |
23+
| [jvm.buffer.memory.limit](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmbuffermemorylimit) | experimental | UpDownCounter | jvm.buffer.pool.name | Maximum memory usage for buffers |
24+
| [jvm.buffer.memory.count](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmbuffermemorycount) | experimental | UpDownCounter | jvm.buffer.pool.name | Buffers count |
25+
26+
## Limitations and unsupported metrics
27+
28+
There are a few limitations to the JVM metrics that are captured through the JMX interface with declarative YAML.
29+
Using the [runtime-telemetry](../../runtime-telemetry) modules with instrumentation allow to capture metrics without those limitations.
30+
31+
[^1]: `jvm.thread.daemon` and `jvm.thread.state` attributes are not supported.
32+
33+
- [jvm.gc.duration](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmgcduration) metric is not supported as it is only exposed through JMX notifications which are not supported with YAML.
34+
- [jvm.cpu.time](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmcputime) metric is not supported yet due to lack of unit conversion, see [#13369](https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/13369) for details.

‎instrumentation/jmx-metrics/library/src/main/resources/jmx/rules/jvm.yaml

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
---
2+
3+
rules:
4+
5+
- bean: java.lang:type=MemoryPool,name=*
6+
prefix: jvm.memory.
7+
type: updowncounter
8+
unit: By
9+
metricAttribute:
10+
jvm.memory.pool.name: param(name)
11+
jvm.memory.type: lowercase(beanattr(Type))
12+
mapping:
13+
# jvm.memory.used
14+
Usage.used:
15+
metric: used
16+
desc: Measure of memory used.
17+
# jvm.memory.committed
18+
Usage.committed:
19+
metric: committed
20+
desc: Measure of memory committed.
21+
# jvm.memory.limit
22+
Usage.max:
23+
metric: limit
24+
desc: Measure of max obtainable memory.
25+
# jvm.memory.init (experimental)
26+
Usage.init:
27+
metric: init
28+
desc: Measure of initial memory requested.
29+
# jvm.memory.used_after_last_gc
30+
# note: metric attribute "jvm.memory.type" will always be "heap" as GC only manages heap
31+
CollectionUsage.used:
32+
metric: used_after_last_gc
33+
desc: Measure of memory used, as measured after the most recent garbage collection event on this pool.
34+
35+
- bean: java.lang:type=Threading
36+
prefix: jvm.thread.
37+
mapping:
38+
# jvm.thread.count
39+
# limitation: 'jvm.thread.daemon' and 'jvm.thread.state' metric attributes are not provided
40+
ThreadCount:
41+
metric: count
42+
type: updowncounter
43+
unit: "{thread}"
44+
desc: Number of executing platform threads.
45+
46+
- bean: java.lang:type=ClassLoading
47+
prefix: jvm.class.
48+
type: updowncounter
49+
unit: "{class}"
50+
mapping:
51+
# jvm.class.loaded
52+
TotalLoadedClassCount:
53+
metric: loaded
54+
desc: Number of classes loaded since JVM start.
55+
# jvm.class.unloaded
56+
UnloadedClassCount:
57+
metric: unloaded
58+
desc: Number of classes unloaded since JVM start.
59+
# jvm.class.count
60+
LoadedClassCount:
61+
metric: count
62+
desc: Number of classes currently loaded.
63+
64+
- bean: java.lang:type=OperatingSystem
65+
prefix: jvm.
66+
dropNegativeValues: true
67+
mapping:
68+
# jvm.cpu.count
69+
AvailableProcessors:
70+
metric: cpu.count
71+
type: updowncounter
72+
unit: "{cpu}"
73+
desc: Number of processors available to the Java virtual machine.
74+
# jvm.cpu.time
75+
ProcessCpuTime:
76+
metric: cpu.time
77+
type: counter
78+
sourceUnit: ns
79+
unit: s
80+
desc: CPU time used by the process as reported by the JVM.
81+
# jvm.cpu.recent_utilization
82+
ProcessCpuLoad:
83+
metric: cpu.recent_utilization
84+
type: gauge
85+
unit: '1'
86+
desc: Recent CPU utilization for the process as reported by the JVM.
87+
# jvm.system.cpu.load_1m (experimental)
88+
SystemLoadAverage:
89+
metric: system.cpu.load_1m
90+
type: gauge
91+
unit: "{run_queue_item}"
92+
desc: Average CPU load of the whole system for the last minute as reported by the JVM.
93+
# jvm.system.cpu.utilization (experimental)
94+
SystemCpuLoad:
95+
metric: system.cpu.utilization
96+
type: gauge
97+
unit: '1'
98+
desc: Recent CPU utilization for the whole system as reported by the JVM.
99+
100+
- bean: java.nio:name=*,type=BufferPool
101+
prefix: jvm.buffer.
102+
type: updowncounter
103+
metricAttribute:
104+
jvm.buffer.pool.name: param(name)
105+
mapping:
106+
# jvm.buffer.memory.used (experimental)
107+
MemoryUsed:
108+
metric: memory.used
109+
unit: By
110+
desc: Measure of memory used by buffers.
111+
# jvm.buffer.memory.limit (experimental)
112+
TotalCapacity:
113+
metric: memory.limit
114+
unit: By
115+
desc: Measure of total memory capacity of buffers.
116+
# jvm.buffer.count (experimental)
117+
Count:
118+
metric: count
119+
unit: "{buffer}"
120+
desc: Number of buffers in the pool.

‎instrumentation/jmx-metrics/library/src/test/java/io/opentelemetry/instrumentation/jmx/rules/JvmTargetSystemTest.java

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
/*
2+
* Copyright The OpenTelemetry Authors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package io.opentelemetry.instrumentation.jmx.rules;
7+
8+
import static io.opentelemetry.instrumentation.jmx.rules.assertions.DataPointAttributes.attribute;
9+
import static io.opentelemetry.instrumentation.jmx.rules.assertions.DataPointAttributes.attributeGroup;
10+
import static io.opentelemetry.instrumentation.jmx.rules.assertions.DataPointAttributes.attributeWithAnyValue;
11+
12+
import io.opentelemetry.instrumentation.jmx.rules.assertions.AttributeMatcher;
13+
import io.opentelemetry.instrumentation.jmx.rules.assertions.AttributeMatcherGroup;
14+
import java.time.Duration;
15+
import java.util.ArrayList;
16+
import java.util.Collections;
17+
import java.util.List;
18+
import org.junit.jupiter.params.ParameterizedTest;
19+
import org.junit.jupiter.params.provider.ValueSource;
20+
import org.testcontainers.containers.GenericContainer;
21+
import org.testcontainers.containers.wait.strategy.Wait;
22+
23+
class JvmTargetSystemTest extends TargetSystemTest {
24+
25+
@ParameterizedTest
26+
@ValueSource(
27+
strings = {
28+
// openj9 image that might have slight differences for JVM metrics
29+
"tomcat:jdk8-adoptopenjdk-openj9",
30+
// basic tomcat image with standard hotspot jdk
31+
"tomcat:9.0"
32+
})
33+
void testJvmMetrics(String image) {
34+
List<String> yamlFiles = Collections.singletonList("jvm.yaml");
35+
36+
yamlFiles.forEach(this::validateYamlSyntax);
37+
38+
List<String> jvmArgs = new ArrayList<>();
39+
jvmArgs.add(javaAgentJvmArgument());
40+
jvmArgs.addAll(javaPropertiesToJvmArgs(otelConfigProperties(yamlFiles)));
41+
42+
// testing with a basic tomcat image as test application to capture JVM metrics
43+
GenericContainer<?> target =
44+
new GenericContainer<>(image)
45+
.withEnv("CATALINA_OPTS", String.join(" ", jvmArgs))
46+
.withStartupTimeout(Duration.ofMinutes(2))
47+
.withExposedPorts(8080)
48+
.waitingFor(Wait.forListeningPorts(8080));
49+
50+
copyFilesToTarget(target, yamlFiles);
51+
52+
startTarget(target);
53+
54+
AttributeMatcher poolNameAttribute = attributeWithAnyValue("jvm.memory.pool.name");
55+
56+
AttributeMatcherGroup heapPoolAttributes =
57+
attributeGroup(attribute("jvm.memory.type", "heap"), poolNameAttribute);
58+
59+
AttributeMatcherGroup nonHeapPoolAttributes =
60+
attributeGroup(attribute("jvm.memory.type", "non_heap"), poolNameAttribute);
61+
62+
AttributeMatcher bufferPoolName = attributeWithAnyValue("jvm.buffer.pool.name");
63+
verifyMetrics(
64+
MetricsVerifier.create()
65+
.add(
66+
"jvm.memory.used",
67+
metric ->
68+
metric
69+
.hasDescription("Measure of memory used.")
70+
.hasUnit("By")
71+
.isUpDownCounter()
72+
.hasDataPointsWithAttributes(heapPoolAttributes, nonHeapPoolAttributes))
73+
.add(
74+
"jvm.memory.committed",
75+
metric ->
76+
metric
77+
.hasDescription("Measure of memory committed.")
78+
.hasUnit("By")
79+
.isUpDownCounter()
80+
.hasDataPointsWithAttributes(heapPoolAttributes, nonHeapPoolAttributes))
81+
.add(
82+
"jvm.memory.limit",
83+
metric ->
84+
metric
85+
.hasDescription("Measure of max obtainable memory.")
86+
.hasUnit("By")
87+
.isUpDownCounter()
88+
.hasDataPointsWithAttributes(heapPoolAttributes, nonHeapPoolAttributes))
89+
.add(
90+
"jvm.memory.init",
91+
metric ->
92+
metric
93+
.hasDescription("Measure of initial memory requested.")
94+
.hasUnit("By")
95+
.isUpDownCounter()
96+
.hasDataPointsWithAttributes(heapPoolAttributes, nonHeapPoolAttributes))
97+
.add(
98+
"jvm.memory.used_after_last_gc",
99+
metric ->
100+
metric
101+
.hasDescription(
102+
"Measure of memory used, as measured after the most recent garbage collection event on this pool.")
103+
.hasUnit("By")
104+
.isUpDownCounter()
105+
// note: there is no GC for non-heap memory
106+
.hasDataPointsWithAttributes(heapPoolAttributes))
107+
.add(
108+
"jvm.thread.count",
109+
metric ->
110+
metric
111+
.hasDescription("Number of executing platform threads.")
112+
.hasUnit("{thread}")
113+
.isUpDownCounter()
114+
.hasDataPointsWithoutAttributes())
115+
.add(
116+
"jvm.class.loaded",
117+
metric ->
118+
metric
119+
.hasDescription("Number of classes loaded since JVM start.")
120+
.hasUnit("{class}")
121+
.isUpDownCounter()
122+
.hasDataPointsWithoutAttributes())
123+
.add(
124+
"jvm.class.unloaded",
125+
metric ->
126+
metric
127+
.hasDescription("Number of classes unloaded since JVM start.")
128+
.hasUnit("{class}")
129+
.isUpDownCounter()
130+
.hasDataPointsWithoutAttributes())
131+
.add(
132+
"jvm.class.count",
133+
metric ->
134+
metric
135+
.hasDescription("Number of classes currently loaded.")
136+
.hasUnit("{class}")
137+
.isUpDownCounter()
138+
.hasDataPointsWithoutAttributes())
139+
.add(
140+
"jvm.cpu.count",
141+
metric ->
142+
metric
143+
.hasDescription(
144+
"Number of processors available to the Java virtual machine.")
145+
.hasUnit("{cpu}")
146+
.isUpDownCounter()
147+
.hasDataPointsWithoutAttributes())
148+
.add(
149+
"jvm.cpu.time",
150+
metric ->
151+
metric
152+
.hasDescription("CPU time used by the process as reported by the JVM.")
153+
.hasUnit("s")
154+
.isCounter()
155+
.hasDataPointsWithoutAttributes())
156+
.add(
157+
"jvm.cpu.recent_utilization",
158+
metric ->
159+
metric
160+
.hasDescription(
161+
"Recent CPU utilization for the process as reported by the JVM.")
162+
.hasUnit("1")
163+
.isGauge()
164+
.hasDataPointsWithoutAttributes())
165+
.add(
166+
"jvm.system.cpu.load_1m",
167+
metric ->
168+
metric
169+
.hasDescription(
170+
"Average CPU load of the whole system for the last minute as reported by the JVM.")
171+
.hasUnit("{run_queue_item}")
172+
.isGauge()
173+
.hasDataPointsWithoutAttributes())
174+
.add(
175+
"jvm.system.cpu.utilization",
176+
metric ->
177+
metric
178+
.hasDescription(
179+
"Recent CPU utilization for the whole system as reported by the JVM.")
180+
.hasUnit("1")
181+
.isGauge()
182+
.hasDataPointsWithoutAttributes())
183+
.add(
184+
"jvm.buffer.memory.used",
185+
metric ->
186+
metric
187+
.hasDescription("Measure of memory used by buffers.")
188+
.hasUnit("By")
189+
.isUpDownCounter()
190+
.hasDataPointsWithOneAttribute(bufferPoolName))
191+
.add(
192+
"jvm.buffer.memory.limit",
193+
metric ->
194+
metric
195+
.hasDescription("Measure of total memory capacity of buffers.")
196+
.hasUnit("By")
197+
.isUpDownCounter()
198+
.hasDataPointsWithOneAttribute(bufferPoolName))
199+
.add(
200+
"jvm.buffer.count",
201+
metric ->
202+
metric
203+
.hasDescription("Number of buffers in the pool.")
204+
.hasUnit("{buffer}")
205+
.isUpDownCounter()
206+
.hasDataPointsWithOneAttribute(bufferPoolName)));
207+
}
208+
}

0 commit comments

Comments
 (0)