Skip to content

Commit 8c9511f

Browse files
edwardnealcheenamalhotraCopilot
authored
Implement performance benchmarks for Always Encrypted scenarios (#4502)
* Create new RAII types for CMK and CEK objects * Designate certain types as compatible with Always Encrypted * Refactor DataTypeReaderRunner * Replace X hardcoded test methods with a single ParamsSource. * Merge DataTypeReaderRunner and its async variant together. * Implement base class, and two child classes - one for AlwaysEncrypted, one for plaintext. * Move the connection open and the table setup logic out of the benchmark code. * Refactor AsyncLargeDataReadRunner This was actually running both sync and async tests. * Implement base class, and two child classes - one for AlwaysEncrypted, one for plaintext. * Expand test to cover CommandBehavior.Default and SequentialAccess. * Include GetFieldValue and GetFieldValueAsync. * Move repeated allocation of the buffer out of the main ReadLargeDataSync_GetBytes benchmark loop. * Documentation/style changes * File-scoped namespaces. * Remove references to removed benchmarks. * Honour MdsPackageVersion in TestCommon package reference PerformanceTests now references TestCommon. In Package mode the perf pipeline pins a released MDS baseline via -p:MdsPackageVersion, but TestCommon ignored that property and resolved the CPM-managed in-development version instead. That version is not published, so restore failed with NU1102 and the mismatch produced an NU1605 downgrade error. Apply the same conditional VersionOverride pattern used by PerformanceTests, including the 7.1.0-preview1.26124.5 transitive dependency pins. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 82c3df83-8f5b-487a-b57a-31d5acb40abe * Fix in-process timeout failure in large data read benchmarks The large data read benchmarks failed with: System.InvalidOperationException: Benchmark Plaintext.ReadLargeDataSync_GetBytes ... takes too long to run. Prefer to use out-of-process toolchains for long-running benchmarks. BenchmarkDotNet's in-process executor aborts a benchmark case after a hard-coded 5 minute default. A single case here reads up to 20 MB per operation across 20 iterations, plus the extra iterations that MemoryDiagnoser and ThreadingDiagnoser each add, so it exceeds that. Switching to an out-of-process toolchain (BenchmarkDotNet's own suggestion) is not viable for this suite. The AppContext switches set in Program.SetupConfigurations - managed SNI, connection pool V2, optimized async behaviour - only apply to the process running Main. An out-of-process toolchain spawns a generated host where they revert to their defaults, so the benchmarks would silently measure the wrong code paths. Instead, add two optional per-benchmark knobs to runnerconfig.jsonc and apply them to the two large data read runners: TimeoutMinutes - overrides the in-process execution timeout (30 minutes here); omitted elsewhere, so other benchmarks keep the BenchmarkDotNet default. RunStrategy - selects the BenchmarkDotNet RunStrategy. The large data read runners use Monitoring, which skips the harness-overhead measurement Throughput performs; that measurement is meaningless when a single operation takes seconds. Their IterationCount also drops from 20 to 5, since network-bound 20 MB reads gain little statistical value from the extra iterations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 82c3df83-8f5b-487a-b57a-31d5acb40abe * Move ReadBufferBytes to Plaintext derived class This was never used by Always Encrypted. * Use the existing SqlDataReader extension methods to flush the result set * Reorder members by visibility * Force SqlDataExtensions to explicitly flush the row data * Replace Params with Arguments This eliminates a number of unnecessary benchmark runs * Lift Always Encrypted definition values to constants * Calculate DefaultValue and EncryptionSupported properties * Remove VersionOverride for TestCommon --------- Co-authored-by: Cheena Malhotra <cmalhotra@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 82c3df83-8f5b-487a-b57a-31d5acb40abe
1 parent f2de4d6 commit 8c9511f

25 files changed

Lines changed: 806 additions & 402 deletions

‎BUILDGUIDE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -494,7 +494,7 @@ The top-level flags control global runner behavior:
494494
| `WaitForProfiler` | Pauses at startup and prints the process ID so you can attach an external profiler (e.g. `dotnet-trace`) before benchmarks run. |
495495
| `UseNativeMemoryAndETWProfiler` | Attaches the `NativeMemoryProfiler` and `EtwProfiler` BenchmarkDotNet diagnosers. Windows only; has no effect on other OSes. |
496496

497-
Some benchmarks (e.g. `DataTypeReaderRunner`, `DataTypeReaderAsyncRunner`) also
497+
Some benchmarks (e.g. `DataTypeReaderRunner`) also
498498
read per-type test values from `datatypes.json` in the `PerformanceTests`
499499
directory. Like `runnerconfig.jsonc`, this file's location can be overridden
500500
with the `DATATYPES_CONFIG` environment variable.

‎eng/pipelines/perf/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -209,7 +209,7 @@ supplies the isolated dedicated host, the tuned SQL instance, and the disjoint c
209209
| Client CPU pin | Pins the benchmark process to `PERF_CLIENT_CPUS` (`taskset` on Linux, `ProcessorAffinity` on Windows). |
210210
| Fail loud | Preflight `SELECT 1` before any pass, **and** a post-pass guard that fails the run if a pass produced **zero** benchmark results — so an empty comparison can never be reported green. |
211211
| Warm-up | Touches the target DB in the preflight to warm the buffer pool / plan cache before the first measured benchmark. |
212-
| Allocator tuning (Linux) | Exports `MALLOC_MMAP_THRESHOLD_=128MiB` and `MALLOC_TRIM_THRESHOLD_=-1` so large-buffer benches (`AsyncLargeDataRead`, `SqlBulkCopy`) stop re-`mmap`ing per iteration. |
212+
| Allocator tuning (Linux) | Exports `MALLOC_MMAP_THRESHOLD_=128MiB` and `MALLOC_TRIM_THRESHOLD_=-1` so large-buffer benches (`LargeDataRead`, `SqlBulkCopy`) stop re-`mmap`ing per iteration. |
213213
| Network tuning (Linux) | Best-effort `sysctl` to widen the ephemeral port range and enable `tcp_tw_reuse` for churn benches (`ConnectionPoolStress`, `ParallelAsyncConnection`). Never fails the run. |
214214
| Diagnostics | Writes `results/diagnostics/`: SQL instance config (MAXDOP, memory, affinity, tempdb files, `@@VERSION`), host CPU topology, and per-pass CPU-clock/thermal telemetry (before/after each pass). |
215215
| Regression gate | `failOnRegression` threads `--fail-on-regression`; only a **candidate-slower** delta past the threshold fails, and in interleaved mode only after best-of-N confirmation. Default off. |

‎eng/pipelines/perf/scripts/run-perf-tests.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -288,7 +288,7 @@ DIAG_DIR="${RESULTS_DIR}/diagnostics"
288288
mkdir -p "${DIAG_DIR}"
289289

290290
# --- §2.8 Allocator tuning (exported so the 'dotnet run' children inherit it) ---------------------
291-
# Large-buffer benches (AsyncLargeDataRead, SqlBulkCopy) re-mmap a big buffer every iteration under
291+
# Large-buffer benches (LargeDataRead, SqlBulkCopy) re-mmap a big buffer every iteration under
292292
# glibc malloc; keep those allocations on the heap and stop trimming freed pages so they are reused,
293293
# which removes a major source of per-iteration variance.
294294
export MALLOC_MMAP_THRESHOLD_="${MALLOC_MMAP_THRESHOLD_:-134217728}" # 128 MiB

‎src/Microsoft.Data.SqlClient/tests/Common/Fixtures/ColumnMasterKeyCertificateFixture.cs‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,17 @@ public ColumnMasterKeyCertificateFixture()
2222

2323
public X509Certificate2? ColumnMasterKeyCertificate { get; }
2424

25+
public string? ColumnMasterKeyCertificatePath { get; }
26+
2527
protected ColumnMasterKeyCertificateFixture(bool createCertificate)
2628
{
2729
if (createCertificate)
2830
{
2931
ColumnMasterKeyCertificate = CreateCertificate(nameof(ColumnMasterKeyCertificate), Array.Empty<string>(), Array.Empty<string>());
3032

3133
AddToStore(ColumnMasterKeyCertificate, StoreLocation.CurrentUser, StoreName.My);
34+
35+
ColumnMasterKeyCertificatePath = $"{StoreLocation.CurrentUser}/{StoreName.My}/{ColumnMasterKeyCertificate.Thumbprint}";
3236
}
3337
}
3438
}

‎src/Microsoft.Data.SqlClient/tests/Common/Fixtures/DatabaseObjects/ColumnEncryptionKey.cs‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
// Licensed to the .NET Foundation under one or more agreements.
2+
// The .NET Foundation licenses this file to you under the MIT license.
3+
// See the LICENSE file in the project root for more information.
4+
5+
using System.Security.Cryptography;
6+
7+
namespace Microsoft.Data.SqlClient.Tests.Common.Fixtures.DatabaseObjects;
8+
9+
/// <summary>
10+
/// A column encryption key, created at the start of its scope and dropped when disposed.
11+
/// </summary>
12+
public sealed class ColumnEncryptionKey : DatabaseObject<ColumnMasterKey>
13+
{
14+
private const int PlaintextKeyLength = 32;
15+
16+
private const string DefinitionTemplate = "CREATE COLUMN ENCRYPTION KEY {0} WITH VALUES" +
17+
" (COLUMN_MASTER_KEY = {1}, ALGORITHM = 'RSA_OAEP', ENCRYPTED_VALUE = 0x{2})";
18+
19+
private ColumnMasterKey ColumnMasterKey => State;
20+
21+
/// <summary>
22+
/// Initializes a new instance of the ColumnEncryptionKey class using the specified SQL connection,
23+
/// name and a column master key.
24+
/// </summary>
25+
/// <param name="connection">The SQL connection used to interact with the database.</param>
26+
/// <param name="namePrefix">The column encryption key name.</param>
27+
/// <param name="cmkOrigin">The column master key which backs this encryption key.</param>
28+
public ColumnEncryptionKey(SqlConnection connection, string namePrefix, ColumnMasterKey cmkOrigin)
29+
: base(connection, GenerateLongName(namePrefix), definition: DefinitionTemplate,
30+
state: cmkOrigin, shouldCreate: true, shouldDrop: true)
31+
{
32+
}
33+
34+
protected override void CreateObject(string definition)
35+
{
36+
string encryptedValue;
37+
38+
using (RandomNumberGenerator rnd = RandomNumberGenerator.Create())
39+
{
40+
byte[] randomPlaintext = new byte[PlaintextKeyLength];
41+
byte[] encryptedPlaintext;
42+
43+
rnd.GetBytes(randomPlaintext);
44+
encryptedPlaintext = ColumnMasterKey.Encrypt(randomPlaintext);
45+
46+
encryptedValue = BitConverter.ToString(encryptedPlaintext).Replace("-", "");
47+
}
48+
49+
definition = string.Format(definition, Name, ColumnMasterKey.Name, encryptedValue);
50+
using SqlCommand createCommand = new(definition, Connection);
51+
52+
createCommand.ExecuteNonQuery();
53+
}
54+
55+
protected override void DropObject()
56+
{
57+
using SqlCommand dropCommand = new($"IF EXISTS (SELECT 1 FROM sys.column_encryption_keys where name = @Name) DROP COLUMN ENCRYPTION KEY {Name}", Connection);
58+
dropCommand.Parameters.AddWithValue("@Name", UnescapedName);
59+
60+
dropCommand.ExecuteNonQuery();
61+
}
62+
}

‎src/Microsoft.Data.SqlClient/tests/Common/Fixtures/DatabaseObjects/ColumnMasterKey.cs‎

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
// Licensed to the .NET Foundation under one or more agreements.
2+
// The .NET Foundation licenses this file to you under the MIT license.
3+
// See the LICENSE file in the project root for more information.
4+
5+
using System.Text;
6+
7+
namespace Microsoft.Data.SqlClient.Tests.Common.Fixtures.DatabaseObjects;
8+
9+
/// <summary>
10+
/// A column master key, created at the start of its scope and dropped when disposed.
11+
/// </summary>
12+
public abstract class ColumnMasterKey : DatabaseObject<ColumnMasterKey.CreationParameters>
13+
{
14+
private const string DefinitionTemplate = "CREATE COLUMN MASTER KEY {0} WITH (KEY_STORE_PROVIDER_NAME = '{1}', KEY_PATH = '{2}'{3})";
15+
16+
public sealed class CreationParameters
17+
{
18+
public SqlColumnEncryptionKeyStoreProvider Provider { get; }
19+
20+
public string ProviderName { get; }
21+
22+
public string KeyPath { get; }
23+
24+
public bool AllowEnclaveComputations { get; }
25+
26+
internal CreationParameters(SqlColumnEncryptionKeyStoreProvider provider,
27+
string providerName,
28+
string keyPath,
29+
bool allowEnclaveComputations)
30+
{
31+
Provider = provider;
32+
ProviderName = providerName;
33+
KeyPath = keyPath;
34+
AllowEnclaveComputations = allowEnclaveComputations;
35+
}
36+
}
37+
38+
protected ColumnMasterKey(SqlConnection connection, string namePrefix, CreationParameters creationParameters)
39+
: base(connection, name: GenerateLongName(namePrefix), definition: DefinitionTemplate,
40+
state: creationParameters, shouldCreate: true, shouldDrop: true)
41+
{
42+
}
43+
44+
protected override void CreateObject(string definition)
45+
{
46+
string enclaveStatement;
47+
48+
if (State.AllowEnclaveComputations)
49+
{
50+
byte[] signature = State.Provider.SignColumnMasterKeyMetadata(State.KeyPath, State.AllowEnclaveComputations);
51+
string signatureString = BitConverter.ToString(signature).Replace("-", "");
52+
53+
enclaveStatement = ", ENCLAVE_COMPUTATIONS (SIGNATURE = 0x" + signatureString + ")";
54+
}
55+
else
56+
{
57+
enclaveStatement = string.Empty;
58+
}
59+
60+
definition = string.Format(definition, Name, State.ProviderName, State.KeyPath, enclaveStatement);
61+
62+
using SqlCommand createCommand = new(definition, Connection);
63+
64+
createCommand.ExecuteNonQuery();
65+
}
66+
67+
protected override void DropObject()
68+
{
69+
using SqlCommand dropCommand = new($"IF EXISTS (SELECT 1 FROM sys.column_master_keys where name = @Name) DROP COLUMN MASTER KEY {Name}", Connection);
70+
dropCommand.Parameters.AddWithValue("@Name", UnescapedName);
71+
72+
dropCommand.ExecuteNonQuery();
73+
}
74+
75+
public byte[] Encrypt(byte[] columnEncryptionKey) =>
76+
State.Provider.EncryptColumnEncryptionKey(State.KeyPath, "RSA_OAEP", columnEncryptionKey);
77+
78+
public byte[] Decrypt(byte[] encryptedColumnEncryptionKey) =>
79+
State.Provider.DecryptColumnEncryptionKey(State.KeyPath, "RSA_OAEP", encryptedColumnEncryptionKey);
80+
}
81+
82+
/// <summary>
83+
/// A column master key backed by a Cryptographic Service Provider. Created at the start of its
84+
/// scope and dropped when disposed.
85+
/// </summary>
86+
public sealed class CspProviderBackedColumnMasterKey : ColumnMasterKey
87+
{
88+
/// <summary>
89+
/// Initializes a new instance of the CspProviderBackedColumnMasterKey class using the specified
90+
/// SQL connection, name and a certificate containing a CSP-backed private key.
91+
/// </summary>
92+
/// <remarks>
93+
/// <para>
94+
/// If a column master key with the specified name already exists, it will be dropped automatically
95+
/// before creation.
96+
/// </para>
97+
/// <para>
98+
/// This column master key will be backed by the <see cref="SqlColumnEncryptionCspProvider"/> class.
99+
/// </para>
100+
/// </remarks>
101+
/// <param name="connection">The SQL connection used to interact with the database.</param>
102+
/// <param name="namePrefix">The column master key name.</param>
103+
/// <param name="cspProvider">The certificate to wrap. Must contain a CSP-backed private key.</param>
104+
/// <param name="allowEnclaveComputations"><c>true</c> to enable enclave computations.</param>
105+
public CspProviderBackedColumnMasterKey(SqlConnection connection, string namePrefix,
106+
CspCertificateFixture cspProvider, bool allowEnclaveComputations)
107+
: base(connection, namePrefix, GenerateCreationParameters(cspProvider, allowEnclaveComputations))
108+
{
109+
}
110+
111+
private static CreationParameters GenerateCreationParameters(CspCertificateFixture cspProvider, bool allowEnclaveComputations) =>
112+
new(provider: new SqlColumnEncryptionCspProvider(),
113+
providerName: SqlColumnEncryptionCspProvider.ProviderName,
114+
cspProvider.CspKeyPath ?? throw new InvalidOperationException("Certificate lacks a CSP key."),
115+
allowEnclaveComputations);
116+
}
117+
118+
/// <summary>
119+
/// A column master key backed by a certificate. Created at the start of its scope and dropped when disposed.
120+
/// </summary>
121+
public sealed class CertificateBackedColumnMasterKey : ColumnMasterKey
122+
{
123+
/// <summary>
124+
/// Initializes a new instance of the CertificateBackedColumnMasterKey class using the specified
125+
/// SQL connection, name and a certificate.
126+
/// </summary>
127+
/// <remarks>
128+
/// <para>
129+
/// If a column master key with the specified name already exists, it will be dropped automatically
130+
/// before creation.
131+
/// </para>
132+
/// <para>
133+
/// This column master key will be backed by the <see cref="SqlColumnEncryptionCertificateStoreProvider"/>
134+
/// class.
135+
/// </para>
136+
/// </remarks>
137+
/// <param name="connection">The SQL connection used to interact with the database.</param>
138+
/// <param name="namePrefix">The column master key name.</param>
139+
/// <param name="cspCertificate">The certificate to wrap. Must contain a private key.</param>
140+
/// <param name="allowEnclaveComputations"><c>true</c> to enable enclave computations.</param>
141+
public CertificateBackedColumnMasterKey(SqlConnection connection, string namePrefix,
142+
CspCertificateFixture cspCertificate, bool allowEnclaveComputations)
143+
: base(connection, namePrefix, GenerateCreationParameters(cspCertificate.CspCertificatePath, allowEnclaveComputations))
144+
{
145+
}
146+
147+
/// <summary>
148+
/// Initializes a new instance of the ColumnMasterKey class using the specified SQL connection,
149+
/// name and a certificate.
150+
/// </summary>
151+
/// <remarks>
152+
/// <para>
153+
/// If a column master key with the specified name already exists, it will be dropped automatically
154+
/// before creation.
155+
/// </para>
156+
/// <para>
157+
/// This column master key will be backed by the <see cref="SqlColumnEncryptionCertificateStoreProvider"/>
158+
/// class.
159+
/// </para>
160+
/// </remarks>
161+
/// <param name="connection">The SQL connection used to interact with the database.</param>
162+
/// <param name="namePrefix">The column master key name.</param>
163+
/// <param name="cmkCertificate">The certificate to wrap. Must contain a private key.</param>
164+
/// <param name="allowEnclaveComputations"><c>true</c> to enable enclave computations.</param>
165+
public CertificateBackedColumnMasterKey(SqlConnection connection, string namePrefix,
166+
ColumnMasterKeyCertificateFixture cmkCertificate, bool allowEnclaveComputations)
167+
: base(connection, namePrefix, GenerateCreationParameters(
168+
cmkCertificate.ColumnMasterKeyCertificatePath
169+
?? throw new InvalidOperationException("Certificate has not been created."),
170+
allowEnclaveComputations))
171+
{
172+
}
173+
174+
private static CreationParameters GenerateCreationParameters(string certificatePath, bool allowEnclaveComputations) =>
175+
new(provider: new SqlColumnEncryptionCertificateStoreProvider(),
176+
providerName: SqlColumnEncryptionCertificateStoreProvider.ProviderName,
177+
certificatePath,
178+
allowEnclaveComputations);
179+
}

‎src/Microsoft.Data.SqlClient/tests/Common/Microsoft.Data.SqlClient.TestCommon.csproj‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,19 @@
3737
<ItemGroup>
3838
<ProjectReference Include="$(RepoRoot)src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj"
3939
Condition="'$(ReferenceType)' != 'Package'" />
40+
<!-- In Package mode the perf pipeline can pin a specific released MDS version (e.g. a -->
41+
<!-- performance baseline) by passing -p:MdsPackageVersion=<version>. That property flows -->
42+
<!-- to every project in the build, so this project must honour it too; otherwise the -->
43+
<!-- CPM-managed (unpublished, in-development) version is used here and restore fails with -->
44+
<!-- NU1102/NU1605 for consumers that pinned an older baseline. Under Central Package -->
45+
<!-- Management a plain Version is ignored, so VersionOverride is used; when -->
46+
<!-- MdsPackageVersion is empty the CPM-managed version applies. The two Includes are -->
47+
<!-- mutually exclusive on whether MdsPackageVersion was provided. -->
4048
<PackageReference Include="Microsoft.Data.SqlClient"
41-
Condition="'$(ReferenceType)' == 'Package'" />
49+
Condition="'$(ReferenceType)' == 'Package' and '$(MdsPackageVersion)' == ''" />
50+
<PackageReference Include="Microsoft.Data.SqlClient"
51+
Condition="'$(ReferenceType)' == 'Package' and '$(MdsPackageVersion)' != ''"
52+
VersionOverride="$(MdsPackageVersion)" />
4253
</ItemGroup>
4354

4455
<!-- References for netfx -->

‎src/Microsoft.Data.SqlClient/tests/Common/SqlDataReaderExtensions.cs‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,8 @@ public static void FlushResultSet(this SqlDataReader dataReader)
4545
{
4646
while (dataReader.Read())
4747
{
48-
// Discard results.
48+
// Read all row data and discard.
49+
_ = dataReader.IsDBNull(0);
4950
}
5051
}
5152

@@ -58,7 +59,8 @@ public static async Task FlushResultSetAsync(this SqlDataReader dataReader)
5859
{
5960
while (await dataReader.ReadAsync())
6061
{
61-
// Discard results.
62+
// Read all row data and discard.
63+
_ = await dataReader.IsDBNullAsync(0);
6264
}
6365
}
6466
}

0 commit comments

Comments
 (0)