[tools] Stress test README updates (#5388)

Author: CodeBlanch

test/OpenTelemetry.Tests.Stress.Logs/README.md

@@ -10,5 +10,23 @@ based on the [OpenTelemetry.Tests.Stress](../OpenTelemetry.Tests.Stress/README.m
 Open a console, run the following command from the current folder:
 
 ```sh
-dotnet run --framework net6.0 --configuration Release
+dotnet run --framework net8.0 --configuration Release
+```
+
+To see command line options available, run the following command from the
+current folder:
+
+```sh
+dotnet run --framework net8.0 --configuration Release -- --help
+```
+
+The help output includes settings and their explanations:
+
+```text
+  -c, --concurrency      The concurrency (maximum degree of parallelism) for the stress test. Default value: Environment.ProcessorCount.
+
+  -p, --internal_port    The Prometheus http listener port where Prometheus will be exposed for retrieving internal metrics while the stress test is running. Set to '0' to
+                         disable. Default value: 9464.
+
+  -d, --duration         The duration for the stress test to run in seconds. If set to '0' or a negative value the stress test will run until canceled. Default value: 0.
 ```

test/OpenTelemetry.Tests.Stress.Metrics/README.md

@@ -5,15 +5,41 @@ This stress test is specifically for Metrics SDK, and is based on the
 
 * [Running the stress test](#running-the-stress-test)
 
-> [!NOTE]
-> To run the stress tests for Histogram, comment out the `Run` method
-for `Counter` and uncomment everything related to `Histogram` in the
-[Program.cs](../OpenTelemetry.Tests.Stress.Metrics/Program.cs).
-
 ## Running the stress test
 
 Open a console, run the following command from the current folder:
 
 ```sh
 dotnet run --framework net8.0 --configuration Release
 ```
+
+To see command line options available, run the following command from the
+current folder:
+
+```sh
+dotnet run --framework net8.0 --configuration Release -- --help
+```
+
+The help output includes settings and their explanations:
+
+```text
+  -t, --type             The metrics stress test type to run. Valid values: [Histogram, Counter]. Default value: Histogram.
+
+  -m, --metrics_port     The Prometheus http listener port where Prometheus will be exposed for retrieving test metrics while the stress test is running. Set to '0' to disable.
+                         Default value: 9185.
+
+  -v, --view             Whether or not a view should be configured to filter tags for the stress test. Default value: False.
+
+  -o, --otlp             Whether or not an OTLP exporter should be added for the stress test. Default value: False.
+
+  -i, --interval         The OTLP exporter export interval in milliseconds. Default value: 5000.
+
+  -e, --exemplars        Whether or not to enable exemplars for the stress test. Default value: False.
+
+  -c, --concurrency      The concurrency (maximum degree of parallelism) for the stress test. Default value: Environment.ProcessorCount.
+
+  -p, --internal_port    The Prometheus http listener port where Prometheus will be exposed for retrieving internal metrics while the stress test is running. Set to '0' to
+                         disable. Default value: 9464.
+
+  -d, --duration         The duration for the stress test to run in seconds. If set to '0' or a negative value the stress test will run until canceled. Default value: 0.
+```

test/OpenTelemetry.Tests.Stress.Traces/README.md

@@ -10,5 +10,23 @@ based on the [OpenTelemetry.Tests.Stress](../OpenTelemetry.Tests.Stress/README.m
 Open a console, run the following command from the current folder:
 
 ```sh
-dotnet run --framework net6.0 --configuration Release
+dotnet run --framework net8.0 --configuration Release
+```
+
+To see command line options available, run the following command from the
+current folder:
+
+```sh
+dotnet run --framework net8.0 --configuration Release -- --help
+```
+
+The help output includes settings and their explanations:
+
+```text
+  -c, --concurrency      The concurrency (maximum degree of parallelism) for the stress test. Default value: Environment.ProcessorCount.
+
+  -p, --internal_port    The Prometheus http listener port where Prometheus will be exposed for retrieving internal metrics while the stress test is running. Set to '0' to
+                         disable. Default value: 9464.
+
+  -d, --duration         The duration for the stress test to run in seconds. If set to '0' or a negative value the stress test will run until canceled. Default value: 0.
 ```

test/OpenTelemetry.Tests.Stress/README.md

@@ -18,26 +18,45 @@
 Open a console, run the following command from the current folder:
 
 ```sh
-dotnet run --framework net6.0 --configuration Release
+dotnet run --framework net8.0 --configuration Release
+```
+
+To see command line options available, run the following command from the
+current folder:
+
+```sh
+dotnet run --framework net8.0 --configuration Release -- --help
+```
+
+The help output includes settings and their explanations:
+
+```text
+  -c, --concurrency      The concurrency (maximum degree of parallelism) for the stress test. Default value: Environment.ProcessorCount.
+
+  -p, --internal_port    The Prometheus http listener port where Prometheus will be exposed for retrieving internal metrics while the stress test is running. Set to '0' to
+                         disable. Default value: 9464.
+
+  -d, --duration         The duration for the stress test to run in seconds. If set to '0' or a negative value the stress test will run until canceled. Default value: 0.
 ```
 
 Once the application started, you will see the performance number updates from
-the console window title.
+the console window title and the console window itself.
+
+While a test is running...
 
-Use the `SPACE` key to toggle the console output, which is off by default.
+* Use the `SPACE` key to toggle the console output, which is on by default.
 
-Use the `ENTER` key to print the latest performance statistics.
+* Use the `ENTER` key to print the latest performance statistics.
 
-Use the `ESC` key to exit the stress test.
+* Use the `ESC` key to exit the stress test.
+
+Example output while a test is running:
 
 ```text
-Running (concurrency = 1), press <Esc> to stop...
-2021-09-28T18:47:17.6807622Z Loops: 17,549,732,467, Loops/Second: 738,682,519, CPU Cycles/Loop: 3
-2021-09-28T18:47:17.8846348Z Loops: 17,699,532,304, Loops/Second: 731,866,438, CPU Cycles/Loop: 3
-2021-09-28T18:47:18.0914577Z Loops: 17,850,498,225, Loops/Second: 730,931,752, CPU Cycles/Loop: 3
-2021-09-28T18:47:18.2992864Z Loops: 18,000,133,808, Loops/Second: 724,029,883, CPU Cycles/Loop: 3
-2021-09-28T18:47:18.5052989Z Loops: 18,150,598,194, Loops/Second: 733,026,161, CPU Cycles/Loop: 3
-2021-09-28T18:47:18.7116733Z Loops: 18,299,461,007, Loops/Second: 724,950,210, CPU Cycles/Loop: 3
+Options: {"Concurrency":20,"PrometheusInternalMetricsPort":9464,"DurationSeconds":0}
+Run OpenTelemetry.Tests.Stress.exe --help to see available options.
+Running (concurrency = 20, internalPrometheusEndpoint = http://localhost:9464/metrics/), press <Esc> to stop, press <Spacebar> to toggle statistics in the console...
+Loops: 17,384,826,748, Loops/Second: 2,375,222,037, CPU Cycles/Loop: 24, RunningTime (Seconds): 7
 ```
 
 The stress test metrics are exposed via
@@ -73,59 +92,91 @@ process_runtime_dotnet_gc_allocations_size_bytes 5485192 1658950184752
 
 ## Writing your own stress test
 
-> [!WARNING]
-> These instructions are out of date and should NOT be followed. They will be
-  updated soon.
-
 Create a simple console application with the following code:
 
 ```csharp
-using System.Runtime.CompilerServices;
+using OpenTelemetry.Tests.Stress;
 
-public partial class Program
+public static class Program
 {
-    public static void Main()
+    public static int Main(string[] args)
     {
-        Stress(concurrency: 10, prometheusPort: 9464);
+        return StressTestFactory.RunSynchronously<MyStressTest>(args);
     }
 
-    [MethodImpl(MethodImplOptions.AggressiveInlining)]
-    protected static void Run()
+    private sealed class MyStressTest : StressTest<StressTestOptions>
     {
-        // add your logic here
+        public MyStressTest(StressTestOptions options)
+            : base(options)
+        {
+        }
+
+        protected override void RunWorkItemInParallel()
+        {
+        }
     }
 }
 ```
 
-Add the Skeleton.cs file to your `*.csproj` file:
+Add the following project reference to the project:
 
 ```xml
-  <ItemGroup>
-    <Compile Include="Skeleton.cs" />
-  </ItemGroup>
+<ProjectReference Include="$(RepoRoot)\test\OpenTelemetry.Tests.Stress\OpenTelemetry.Tests.Stress.csproj" />
 ```
 
-Add the following packages to the project:
+Now you are ready to run your own stress test. Add test logic in the
+`RunWorkItemInParallel` method to measure performance.
 
-```shell
-dotnet add package OpenTelemetry.Exporter.Prometheus --prerelease
-dotnet add package OpenTelemetry.Instrumentation.Runtime --prerelease
-```
+To define custom options create an options class which derives from
+`StressTestOptions`:
 
-Now you are ready to run your own stress test.
+```csharp
+using CommandLine;
+using OpenTelemetry.Tests.Stress;
+
+public static class Program
+{
+    public static int Main(string[] args)
+    {
+        return StressTestFactory.RunSynchronously<MyStressTest, MyStressTestOptions>(args);
+    }
+
+    private sealed class MyStressTest : StressTest<MyStressTestOptions>
+    {
+        public MyStressTest(MyStressTestOptions options)
+            : base(options)
+        {
+        }
+
+        protected override void RunWorkItemInParallel()
+        {
+            // Use this.Options here to access options supplied
+            // on the command line.
+        }
+    }
+
+    private sealed class MyStressTestOptions : StressTestOptions
+    {
+        [Option('r', "rate", HelpText = "Add help text here for the rate option. Default value: 0.", Required = false)]
+        public int Rate { get; set; } = 0;
+    }
+}
+```
 
 Some useful notes:
 
-* You can specify the concurrency using `Stress(concurrency: {concurrency
-  number})`, the default value is the number of CPU cores. Keep in mind that
-  concurrency level does not equal to the number of threads.
-* You can specify a local PrometheusExporter listening port using
-  `Stress(prometheusPort: {port number})`, the default value is `0`, which will
-  turn off the PrometheusExporter.
-* You want to put `[MethodImpl(MethodImplOptions.AggressiveInlining)]` on
-  `Run()`, this helps to reduce extra flushes on the CPU instruction cache.
-* You might want to run the stress test under `Release` mode rather than `Debug`
-  mode.
+* It is generally best practice to run the stress test for code compiled in
+  `Release` configuration rather than `Debug` configuration. `Debug` builds
+  typically are not optimized and contain extra code which will change the
+  performance characteristics of the logic under test. The stress test will
+  write a warning message to the console when starting if compiled with `Debug`
+  configuration.
+* You can specify the concurrency using `-c` or `--concurrency` command line
+  argument, the default value if not specified is the number of CPU cores. Keep
+  in mind that concurrency level does not equal to the number of threads.
+* You can use the duration `-d` or `--duration` command line argument to run the
+  stress test for a specific time period. This is useful when comparing changes
+  across multiple runs.
 
 ## Understanding the results
 
@@ -134,4 +185,6 @@ Some useful notes:
   sliding window of few hundreds of milliseconds.
 * `CPU Cycles/Loop` represents the average CPU cycles for each `Run()`
   invocation, based on a small sliding window of few hundreds of milliseconds.
-* `Runaway Time` represents the runaway time (seconds) since the test started.
+* `Total Running Time` represents the running time (seconds) since the test started.
+* `GC Total Allocated Bytes` (not available on .NET Framework) shows the total
+  amount of memory allocated while the test was running.