Skip to content

Commit

Permalink
doc: dotnet tracing example
Browse files Browse the repository at this point in the history
  • Loading branch information
alsoba13 committed Feb 5, 2025
1 parent 2a310e1 commit 37cd6bb
Show file tree
Hide file tree
Showing 19 changed files with 635 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,6 @@ Refer to the [Tempo data source configuration documentation](https://grafana.com

## Examples

Check out the [examples](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) directory for a complete demo application of span profiles in multiple languages.
Check out these demo applications for span profiles:
- [.NET example](https://github.com/grafana/pyroscope/tree/main/examples/tracing/dotnet)
- [Other examples](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) in multiple languages
1 change: 1 addition & 0 deletions examples/tracing/dotnet/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.so
48 changes: 48 additions & 0 deletions examples/tracing/dotnet/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
ARG SDK_VERSION=8.0
# The build images takes an SDK image of the buildplatform, so the platform the build is running on.
FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:$SDK_VERSION AS build

ARG TARGETPLATFORM
ARG BUILDPLATFORM
ARG SDK_VERSION

WORKDIR /dotnet

ADD example .

# Set the target framework to SDK_VERSION
RUN sed -i -E 's|<TargetFramework>.*</TargetFramework>|<TargetFramework>net'$SDK_VERSION'</TargetFramework>|' Example.csproj

# We hardcode linux-x64 here, as the profiler doesn't support any other platform
RUN dotnet publish -o . --framework net$SDK_VERSION --runtime linux-x64 --no-self-contained

# This fetches the SDK
FROM --platform=linux/amd64 pyroscope/pyroscope-dotnet:0.9.2-glibc AS sdk

# Runtime only image of the targetplatfrom, so the platform the image will be running on.
FROM --platform=linux/amd64 mcr.microsoft.com/dotnet/aspnet:$SDK_VERSION

WORKDIR /dotnet

COPY --from=sdk /Pyroscope.Profiler.Native.so ./Pyroscope.Profiler.Native.so
COPY --from=sdk /Pyroscope.Linux.ApiWrapper.x64.so ./Pyroscope.Linux.ApiWrapper.x64.so
COPY --from=build /dotnet/ ./


ENV CORECLR_ENABLE_PROFILING=1
ENV CORECLR_PROFILER={BD1A650D-AC5D-4896-B64F-D6FA25D6B26A}
ENV CORECLR_PROFILER_PATH=/dotnet/Pyroscope.Profiler.Native.so
ENV LD_PRELOAD=/dotnet/Pyroscope.Linux.ApiWrapper.x64.so

ENV PYROSCOPE_APPLICATION_NAME=rideshare.dotnet.push.app
ENV PYROSCOPE_SERVER_ADDRESS=http://pyroscope:4040
ENV PYROSCOPE_LOG_LEVEL=debug
ENV PYROSCOPE_PROFILING_ENABLED=1
ENV PYROSCOPE_PROFILING_ALLOCATION_ENABLED=true
ENV PYROSCOPE_PROFILING_CONTENTION_ENABLED=true
ENV PYROSCOPE_PROFILING_EXCEPTION_ENABLED=true
ENV PYROSCOPE_PROFILING_HEAP_ENABLED=true
ENV RIDESHARE_LISTEN_PORT=5000


CMD sh -c "ASPNETCORE_URLS=http://*:${RIDESHARE_LISTEN_PORT} exec dotnet /dotnet/example.dll"
10 changes: 10 additions & 0 deletions examples/tracing/dotnet/Dockerfile.load-generator
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM python:3.9

RUN pip3 install requests

COPY load-generator.py ./load-generator.py

ENV PYTHONUNBUFFERED=1

CMD [ "python", "load-generator.py" ]

53 changes: 53 additions & 0 deletions examples/tracing/dotnet/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Span Profiles with Grafana Tempo and Pyroscope

The docker compose consists of:
- The .NET Rideshare App
- Tempo
- Pyroscope
- Grafana

The `rideshare` app generate traces and profiling data that should be available in Grafana.
Datasources for Pyroscope and Tempo are provisioned automatically.

### Build and run

The project can be run locally with the following commands:

```shell
# (optionally) pull latest pyroscope and grafana images:
docker pull grafana/pyroscope:latest
docker pull grafana/grafana:latest

# build and run the example
docker compose up --build
```

Navigate to the [Explore page](http://localhost:3000/explore?schemaVersion=1&panes=%7B%22f36%22:%7B%22datasource%22:%22tempo%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22datasource%22:%7B%22type%22:%22tempo%22,%22uid%22:%22tempo%22%7D,%22queryType%22:%22traceqlSearch%22,%22limit%22:20,%22tableType%22:%22traces%22,%22filters%22:%5B%7B%22id%22:%22e73a615e%22,%22operator%22:%22%3D%22,%22scope%22:%22span%22%7D,%7B%22id%22:%22service-name%22,%22tag%22:%22service.name%22,%22operator%22:%22%3D%22,%22scope%22:%22resource%22,%22value%22:%5B%22rideshare.dotnet.push.app%22%5D,%22valueType%22:%22string%22%7D%5D,%22query%22:%22%7Bresource.service.name%3D%5C%22rideshare.dotnet.push.app%5C%22%7D%22%7D%5D,%22range%22:%7B%22from%22:%22now-15m%22,%22to%22:%22now%22%7D%7D%7D&orgId=1), select a trace and click on a span that has a linked profile:

![image](https://github.com/grafana/otel-profiling-go/assets/12090599/31e33cd1-818b-4116-b952-c9ec7b1fb593)

By default, only the root span gets labeled (the first span created locally): such spans are marked with the _link_ icon
and have the `pyroscope.profile.id` attribute set to the corresponding span ID.
Please note that presence of the attribute does not necessarily
indicate that the span has a profile: stack trace samples might not be collected, if the utilized CPU time is
less than the sample interval (10ms).

### Instrumentation

The `rideshare` demo application is instrumented with Pyroscope: [Pyroscope .NET Agent](https://github.com/grafana/pyroscope-dotnet)

### Grafana Tempo configuration

In order to correlate trace spans with profiling data, the Tempo datasource should have the following configured:
- The profiling data source
- Tags to use when making profiling queries

![image](https://github.com/grafana/pyroscope/assets/12090599/380ac574-a298-440d-acfb-7bc0935a3a7c)

While tags are optional, configuring them is highly recommended for optimizing query performance.
In our example, we configured the `service.name` tag for use in Pyroscope queries as the `service_name` label.
This configuration restricts the data set for lookup, ensuring that queries remain
consistently fast. Note that the tags you configure must be present in the span attributes or resources
for a trace to profiles span link to appear.

Please refer to our [documentation](https://grafana.com/docs/grafana/next/datasources/tempo/configure-tempo-data-source/#trace-to-profiles) for more details.
69 changes: 69 additions & 0 deletions examples/tracing/dotnet/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
services:
pyroscope:
image: grafana/pyroscope
ports:
- "4040:4040"

us-east:
ports:
- "5000"
environment: &env
OTLP_URL: tempo:4318
OTEL_TRACES_EXPORTER: otlp
OTEL_EXPORTER_OTLP_ENDPOINT: http://tempo:4317
OTEL_SERVICE_NAME: rideshare.dotnet.push.app
OTEL_METRICS_EXPORTER: none
OTEL_TRACES_SAMPLER: always_on
OTEL_PROPAGATORS: tracecontext
REGION: us-east
PYROSCOPE_LABELS: region=us-east
PYROSCOPE_SERVER_ADDRESS: http://pyroscope:4040
build:
context: .
eu-north:
ports:
- "5000"
environment:
<<: *env
REGION: eu-north
build:
context: .
ap-south:
ports:
- "5000"
environment:
<<: *env
REGION: ap-south
build:
context: .

load-generator:
build:
context: .
dockerfile: Dockerfile.load-generator

grafana:
image: grafana/grafana:latest
environment:
- GF_INSTALL_PLUGINS=grafana-pyroscope-app
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
- GF_AUTH_DISABLE_LOGIN_FORM=true
- GF_FEATURE_TOGGLES_ENABLE=traceToProfiles tracesEmbeddedFlameGraph
volumes:
- ./grafana-provisioning:/etc/grafana/provisioning
ports:
- "3000:3000"

tempo:
image: grafana/tempo:latest
command: [ "-config.file=/etc/tempo.yml" ]
volumes:
- ./tempo/tempo.yml:/etc/tempo.yml
ports:
- "14268:14268" # jaeger ingest
- "3200:3200" # tempo
- "9095:9095" # tempo grpc
- "4317:4317" # otlp grpc
- "4318:4318" # otlp http
- "9411:9411" # zipkin
2 changes: 2 additions & 0 deletions examples/tracing/dotnet/example/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
bin/
obj/
16 changes: 16 additions & 0 deletions examples/tracing/dotnet/example/BikeService.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
namespace Example;

internal class BikeService
{
private readonly OrderService _orderService;

public BikeService(OrderService orderService)
{
_orderService = orderService;
}

public void Order(int searchRadius)
{
_orderService.FindNearestVehicle(searchRadius, "bike");
}
}
16 changes: 16 additions & 0 deletions examples/tracing/dotnet/example/CarService.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
namespace Example;

internal class CarService
{
private readonly OrderService _orderService;

public CarService(OrderService orderService)
{
_orderService = orderService;
}

public void Order(int searchRadius)
{
_orderService.FindNearestVehicle(searchRadius, "car");
}
}
17 changes: 17 additions & 0 deletions examples/tracing/dotnet/example/Example.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<AssemblyName>example</AssemblyName>
<OutputType>Exe</OutputType>
<PackageId>example</PackageId>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="OpenTelemetry.Exporter.Console" Version="1.8.0" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.8.0" />
<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.8.0" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.8.1" />
<PackageReference Include="Pyroscope.OpenTelemetry" Version="0.2.0" />
<PackageReference Include="System.Diagnostics.DiagnosticSource" Version="8.0.1" />
</ItemGroup>
</Project>
4 changes: 4 additions & 0 deletions examples/tracing/dotnet/example/Folder.DotSettings.user
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<wpf:ResourceDictionary xml:space="preserve" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:s="clr-namespace:System;assembly=mscorlib" xmlns:ss="urn:shemas-jetbrains-com:settings-storage-xaml" xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
<s:String x:Key="/Default/Environment/AssemblyExplorer/XmlDocument/@EntryValue">&lt;AssemblyExplorer&gt;
&lt;Assembly Path="/home/korniltsev/.nuget/packages/pyroscope/0.4.0/lib/net6.0/Pyroscope.dll" /&gt;
&lt;/AssemblyExplorer&gt;</s:String></wpf:ResourceDictionary>
57 changes: 57 additions & 0 deletions examples/tracing/dotnet/example/OrderService.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
using System;

namespace Example;

internal class OrderService
{
public void FindNearestVehicle(long searchRadius, string vehicle)
{
lock (_lock)
{
var labels = Pyroscope.LabelSet.Empty.BuildUpon()
.Add("vehicle", vehicle)
.Build();
Pyroscope.LabelsWrapper.Do(labels, () =>
{
for (long i = 0; i < searchRadius * 1000000000; i++)
{
}

if (vehicle.Equals("car"))
{
CheckDriverAvailability(labels, searchRadius);
}
});
}
}

private readonly object _lock = new();

private static void CheckDriverAvailability(Pyroscope.LabelSet ctx, long searchRadius)
{
var region = System.Environment.GetEnvironmentVariable("REGION") ?? "unknown_region";
ctx = ctx.BuildUpon()
.Add("driver_region", region)
.Build();
Pyroscope.LabelsWrapper.Do(ctx, () =>
{
for (long i = 0; i < searchRadius * 1000000000; i++)
{
}

var now = DateTime.Now.Minute % 2 == 0;
var forceMutexLock = DateTime.Now.Minute % 2 == 0;
if ("eu-north".Equals(region) && forceMutexLock)
{
MutexLock(searchRadius);
}
});
}

private static void MutexLock(long searchRadius)
{
for (long i = 0; i < 30 * searchRadius * 1000000000; i++)
{
}
}
}
Loading

0 comments on commit 37cd6bb

Please sign in to comment.