Skip to content

Commit ddef8d6

Browse files
pierantoniomerlinoMMaiero
pierantoniomerlino
and
MMaiero
authored
feat(ai): First implementation of TritonServer metrics feature (#5687)
* First implementation of TritonServer metrics feature Signed-off-by: pierantoniomerlino <[email protected]> * Reverted compatibility version to 1.8 Signed-off-by: pierantoniomerlino <[email protected]> * Reverted compatibility version to 1.8 Signed-off-by: pierantoniomerlino <[email protected]> * Added interval property; several improvements Signed-off-by: pierantoniomerlino <[email protected]> * Added metrics support for native and remote server versions Signed-off-by: pierantoniomerlino <[email protected]> * Removed optional configs used for debug Signed-off-by: pierantoniomerlino <[email protected]> * Added server address to metrics connection Signed-off-by: pierantoniomerlino <[email protected]> * Fixed tests Signed-off-by: pierantoniomerlino <[email protected]> * Added tests Signed-off-by: pierantoniomerlino <[email protected]> * Added more tests Signed-off-by: pierantoniomerlino <[email protected]> * Updated copyright and added comments Signed-off-by: pierantoniomerlino <[email protected]> * Fixed stream usage Signed-off-by: pierantoniomerlino <[email protected]> * Syntactic sugar for make sonar happy Signed-off-by: pierantoniomerlino <[email protected]> * Simplified triton server properties Signed-off-by: pierantoniomerlino <[email protected]> * Updated tests Signed-off-by: pierantoniomerlino <[email protected]> * Increased read and connection timeout Signed-off-by: pierantoniomerlino <[email protected]> * Update kura/org.eclipse.kura.ai.triton.server/OSGI-INF/metatype/org.eclipse.kura.ai.triton.server.TritonServerContainerService.xml Co-authored-by: Matteo Maiero <[email protected]> * Let's do some TDD... Signed-off-by: pierantoniomerlino <[email protected]> * Removed raw metrics api; fixed copyright Signed-off-by: pierantoniomerlino <[email protected]> * Updated metatypes; removed getRawMetrics implementation Signed-off-by: pierantoniomerlino <[email protected]> * Deleted old implementation of TritonServer Signed-off-by: pierantoniomerlino <[email protected]> * Added GPU metrics retrival Signed-off-by: pierantoniomerlino <[email protected]> * Added statistics retrival Signed-off-by: pierantoniomerlino <[email protected]> * Added missing copyright Signed-off-by: pierantoniomerlino <[email protected]> * Updated metrics names Signed-off-by: pierantoniomerlino <[email protected]> * Updated metric names Signed-off-by: pierantoniomerlino <[email protected]> * Fixed tests Signed-off-by: pierantoniomerlino <[email protected]> * Fixed old tests Signed-off-by: pierantoniomerlino <[email protected]> * Replaced statistics parsing method and updated tests Signed-off-by: pierantoniomerlino <[email protected]> * Fixed copyrights Signed-off-by: pierantoniomerlino <[email protected]> * Fixed copyrights again Signed-off-by: pierantoniomerlino <[email protected]> * Refactored ModelStatistis parser Signed-off-by: pierantoniomerlino <[email protected]> * Updated statistics parser using protobuf JsonFormat Signed-off-by: pierantoniomerlino <[email protected]> * Removed spaces in copyrights header Signed-off-by: pierantoniomerlino <[email protected]> * Added missing dependency on test Signed-off-by: pierantoniomerlino <[email protected]> * Fixed copyright in test pom Signed-off-by: pierantoniomerlino <[email protected]> --------- Signed-off-by: pierantoniomerlino <[email protected]> Co-authored-by: Matteo Maiero <[email protected]>
1 parent 35e31e2 commit ddef8d6

File tree

37 files changed

+1705
-392
lines changed

37 files changed

+1705
-392
lines changed

‎kura/org.eclipse.kura.ai.triton.server/META-INF/MANIFEST.MF‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Bundle-Name: org.eclipse.kura.ai.triton.server
44
Bundle-SymbolicName: org.eclipse.kura.ai.triton.server;singleton:=true
55
Bundle-Version: 2.0.0.qualifier
66
Bundle-Vendor: Eclipse Kura
7-
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=17))"
7+
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
88
Service-Component: OSGI-INF/*.xml
99
Bundle-ClassPath: .,
1010
lib/grpc-api.jar,
@@ -14,13 +14,16 @@ Bundle-ClassPath: .,
1414
lib/grpc-protobuf.jar,
1515
lib/grpc-protobuf-lite.jar,
1616
lib/grpc-stub.jar,
17-
lib/perfmark-api.jar
17+
lib/perfmark-api.jar,
18+
lib/protobuf-java-util.jar
1819
Bundle-ActivationPolicy: lazy
1920
Import-Package: com.google.common.base;version="25.0.0",
2021
com.google.common.collect;version="25.0.0",
2122
com.google.common.io;version="25.0.0",
2223
com.google.common.util.concurrent;version="25.0.0",
2324
com.google.gson;version="2.7.0",
25+
com.google.gson.annotations;version="2.9.0",
26+
com.google.gson.reflect;version="[2.9.0,3.0.0)",
2427
com.google.protobuf;version="3.19.3",
2528
org.apache.commons.io;version="2.4.0",
2629
org.apache.commons.io.filefilter;version="2.11.0",
@@ -37,10 +40,10 @@ Import-Package: com.google.common.base;version="25.0.0",
3740
org.bouncycastle.util.encoders;version="1.78.1",
3841
org.bouncycastle.util.io;version="1.78.1",
3942
org.eclipse.kura;version="[1.0,2.0)",
40-
org.eclipse.kura.ai.inference;version="[1.0,1.1)",
43+
org.eclipse.kura.ai.inference;version="[1.1,1.2)",
4144
org.eclipse.kura.configuration;version="[1.1,2.0)",
42-
org.eclipse.kura.core.linux.executor;version="[1.0,2.0)",
4345
org.eclipse.kura.container.orchestration;version="[1.3,2.0)",
46+
org.eclipse.kura.core.linux.executor;version="[1.0,2.0)",
4447
org.eclipse.kura.crypto;version="[1.0,2.0)",
4548
org.eclipse.kura.executor;version="[1.0,2.0)",
4649
org.eclipse.kura.type;version="[1.1,2.0)",

‎kura/org.eclipse.kura.ai.triton.server/OSGI-INF/metatype/org.eclipse.kura.ai.triton.server.TritonServerContainerService.xml‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<!--
33
4-
Copyright (c) 2022, 2024 Eurotech and/or its affiliates and others
4+
Copyright (c) 2022, 2025 Eurotech and/or its affiliates and others
55
66
This program and the accompanying materials are made
77
available under the terms of the Eclipse Public License 2.0
@@ -90,7 +90,7 @@
9090
default=""
9191
description="A semi-colon separated list of configuration for the backends. i.e. tensorflow,version=2;tensorflow,allow-soft-placement=false">
9292
</AD>
93-
93+
9494
<AD id="container.memory"
9595
name="Memory"
9696
description="The maximum amount of memory the container can use in bytes. Set it as a positive integer, optionally followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, or gigabytes.
@@ -158,6 +158,15 @@
158158
default="4194304"
159159
min="1">
160160
</AD>
161+
162+
<AD id="enable.metrics"
163+
name="Enable Triton Server Metrics"
164+
type="Boolean"
165+
cardinality="0"
166+
required="true"
167+
default="true"
168+
description="Enable the Triton Server Metrics feature. This property allows to retrieve models and GPU metrics from an instance of Triton Server.">
169+
</AD>
161170

162171
</OCD>
163172
<Designate factoryPid="org.eclipse.kura.ai.triton.server.TritonServerContainerService">

‎kura/org.eclipse.kura.ai.triton.server/OSGI-INF/metatype/org.eclipse.kura.ai.triton.server.TritonServerNativeService.xml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<!--
33
4-
Copyright (c) 2022 Eurotech and/or its affiliates and others
4+
Copyright (c) 2022, 2025 Eurotech and/or its affiliates and others
55
66
This program and the accompanying materials are made
77
available under the terms of the Eclipse Public License 2.0
@@ -95,6 +95,15 @@
9595
default="4194304"
9696
min="1">
9797
</AD>
98+
99+
<AD id="enable.metrics"
100+
name="Enable Triton Server Metrics"
101+
type="Boolean"
102+
cardinality="0"
103+
required="true"
104+
default="true"
105+
description="Enable the Triton Server Metrics feature. This property allows to retrieve models and GPU metrics from an instance of Triton Server.">
106+
</AD>
98107

99108
</OCD>
100109
<Designate factoryPid="org.eclipse.kura.ai.triton.server.TritonServerNativeService">

‎kura/org.eclipse.kura.ai.triton.server/OSGI-INF/metatype/org.eclipse.kura.ai.triton.server.TritonServerService.xml‎

Lines changed: 0 additions & 121 deletions
This file was deleted.

‎kura/org.eclipse.kura.ai.triton.server/OSGI-INF/org.eclipse.kura.ai.triton.server.TritonServerContainerService.xml‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,23 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<!--
33
4-
Copyright (c) 2022 Eurotech and/or its affiliates and others
4+
Copyright (c) 2022, 2025 Eurotech and/or its affiliates and others
55
6-
This program and the accompanying materials are made
7-
available under the terms of the Eclipse Public License 2.0
8-
which is available at https://www.eclipse.org/legal/epl-2.0/
6+
This program and the accompanying materials are made
7+
available under the terms of the Eclipse Public License 2.0
8+
which is available at https://www.eclipse.org/legal/epl-2.0/
99
10-
SPDX-License-Identifier: EPL-2.0
10+
SPDX-License-Identifier: EPL-2.0
1111
12-
Contributors:
13-
Eurotech
12+
Contributors:
13+
Eurotech
1414
1515
-->
1616
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" activate="activate" configuration-policy="require" deactivate="deactivate" enabled="true" immediate="true" modified="updated" name="org.eclipse.kura.ai.triton.server.TritonServerContainerService">
1717
<implementation class="org.eclipse.kura.ai.triton.server.TritonServerServiceContainerImpl"/>
1818
<service>
1919
<provide interface="org.eclipse.kura.configuration.ConfigurableComponent"/>
20+
<provide interface="org.eclipse.kura.ai.inference.InferenceEngineMetricsService"/>
2021
<provide interface="org.eclipse.kura.ai.inference.InferenceEngineService"/>
2122
</service>
2223
<property name="service.pid" type="String" value="org.eclipse.kura.ai.triton.server.TritonServerContainerService"/>

‎kura/org.eclipse.kura.ai.triton.server/OSGI-INF/org.eclipse.kura.ai.triton.server.TritonServerNativeService.xml‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,24 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<!--
33
4-
Copyright (c) 2022 Eurotech and/or its affiliates and others
4+
Copyright (c) 2022, 2025 Eurotech and/or its affiliates and others
55
6-
This program and the accompanying materials are made
7-
available under the terms of the Eclipse Public License 2.0
8-
which is available at https://www.eclipse.org/legal/epl-2.0/
6+
This program and the accompanying materials are made
7+
available under the terms of the Eclipse Public License 2.0
8+
which is available at https://www.eclipse.org/legal/epl-2.0/
99
10-
SPDX-License-Identifier: EPL-2.0
10+
SPDX-License-Identifier: EPL-2.0
1111
12-
Contributors:
13-
Eurotech
12+
Contributors:
13+
Eurotech
1414
1515
-->
1616
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" activate="activate" configuration-policy="require" deactivate="deactivate" enabled="true" immediate="true" modified="updated" name="org.eclipse.kura.ai.triton.server.TritonServerNativeService">
1717
<implementation class="org.eclipse.kura.ai.triton.server.TritonServerServiceNativeImpl"/>
1818
<service>
1919
<provide interface="org.eclipse.kura.configuration.ConfigurableComponent"/>
2020
<provide interface="org.eclipse.kura.ai.inference.InferenceEngineService"/>
21+
<provide interface="org.eclipse.kura.ai.inference.InferenceEngineMetricsService"/>
2122
</service>
2223
<property name="service.pid" type="String" value="org.eclipse.kura.ai.triton.server.TritonServerNativeService"/>
2324
<reference bind="setCommandExecutorService" cardinality="1..1" interface="org.eclipse.kura.executor.PrivilegedExecutorService" name="PrivilegedExecutorService" policy="static" />

‎kura/org.eclipse.kura.ai.triton.server/OSGI-INF/org.eclipse.kura.ai.triton.server.TritonServerRemoteService.xml‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,24 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<!--
33
4-
Copyright (c) 2022 Eurotech and/or its affiliates and others
4+
Copyright (c) 2022, 2025 Eurotech and/or its affiliates and others
55
6-
This program and the accompanying materials are made
7-
available under the terms of the Eclipse Public License 2.0
8-
which is available at https://www.eclipse.org/legal/epl-2.0/
6+
This program and the accompanying materials are made
7+
available under the terms of the Eclipse Public License 2.0
8+
which is available at https://www.eclipse.org/legal/epl-2.0/
99
10-
SPDX-License-Identifier: EPL-2.0
10+
SPDX-License-Identifier: EPL-2.0
1111
12-
Contributors:
13-
Eurotech
12+
Contributors:
13+
Eurotech
1414
1515
-->
1616
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" activate="activate" configuration-policy="require" deactivate="deactivate" enabled="true" immediate="true" modified="updated" name="org.eclipse.kura.ai.triton.server.TritonServerRemoteService">
1717
<implementation class="org.eclipse.kura.ai.triton.server.TritonServerServiceRemoteImpl"/>
1818
<service>
1919
<provide interface="org.eclipse.kura.configuration.ConfigurableComponent"/>
2020
<provide interface="org.eclipse.kura.ai.inference.InferenceEngineService"/>
21+
<provide interface="org.eclipse.kura.ai.inference.InferenceEngineMetricsService"/>
2122
</service>
2223
<property name="service.pid" type="String" value="org.eclipse.kura.ai.triton.server.TritonServerRemoteService"/>
2324
<reference bind="setCommandExecutorService" cardinality="1..1" interface="org.eclipse.kura.executor.PrivilegedExecutorService" name="PrivilegedExecutorService" policy="static" />

‎kura/org.eclipse.kura.ai.triton.server/OSGI-INF/org.eclipse.kura.ai.triton.server.TritonServerService.xml‎

Lines changed: 0 additions & 25 deletions
This file was deleted.

‎kura/org.eclipse.kura.ai.triton.server/build.properties‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ bin.includes = .,\
2222
lib/grpc-protobuf.jar,\
2323
lib/grpc-protobuf-lite.jar,\
2424
lib/grpc-stub.jar,\
25-
lib/perfmark-api.jar
25+
lib/perfmark-api.jar,\
26+
lib/protobuf-java-util.jar
2627
source.. = src/main/java/,\
2728
target/generated-sources/protobuf/java/,\
2829
target/generated-sources/protobuf/grpc-java/

‎kura/org.eclipse.kura.ai.triton.server/pom.xml‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,13 @@
22
<!--
33
44
Copyright (c) 2022, 2025 Eurotech and/or its affiliates and others
5-
5+
66
This program and the accompanying materials are made
77
available under the terms of the Eclipse Public License 2.0
88
which is available at https://www.eclipse.org/legal/epl-2.0/
9-
9+
1010
SPDX-License-Identifier: EPL-2.0
11-
11+
1212
Contributors:
1313
Eurotech
1414
@@ -87,6 +87,11 @@
8787
<artifactId>perfmark-api</artifactId>
8888
<version>0.26.0</version>
8989
</artifactItem>
90+
<artifactItem>
91+
<groupId>com.google.protobuf</groupId>
92+
<artifactId>protobuf-java-util</artifactId>
93+
<version>3.25.3</version>
94+
</artifactItem>
9095
</artifactItems>
9196
</configuration>
9297
<goals>

0 commit comments

Comments
 (0)