Tests | Manual Test "Set" Trait by benrr101 · Pull Request #4071 · dotnet/SqlClient

benrr101 · 2026-03-21T00:53:51Z

Description

This PR looks big, but it's actually really small. This updates the manual tests project to include all files in all builds, and moves the TestSet build property to a trait. Filtering of test sets can now be done via the filter command line argument.

Why is this good?

Uses existing mechanics to filter test sets, instead of custom baked things.
No surprises because files were not included
We can use "compiledefaultitems" in the project - no need to update the csproj when adding/removing tests.

Build2.proj now forwards the TestSet parameter to the filter argument, combining it with other filters that were provided. Thus, no changes to the pipelines were necessary.

🤖

Codex did the batch update of the test files (though it applied changes to files that didn't have any tests, so I had to back those changes out).

Testing

Everything works locally, so it should be good in PR validation.

paulmedynski

Looks like Copilot found some bugs to address. I have a question about backwards compatibility with build.proj.

paulmedynski

I agree with the Copilot comments.

Clearing since my changes have been implemented. Waiting for author to reply before approving.

codecov · 2026-05-05T17:35:27Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.05%. Comparing base (bfbdd30) to head (229711c).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4071      +/-   ##
==========================================
- Coverage   66.69%   64.05%   -2.64%     
==========================================
  Files         284      280       -4     
  Lines       43238    66160   +22922     
==========================================
+ Hits        28836    42380   +13544     
- Misses      14402    23780    +9378

Flag	Coverage Δ
CI-SqlClient	`?`
PR-SqlClient-Project	`64.05% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

apoorvdeshmukh

@benrr101 Can you resolve the branch conflicts? I think this one is good to be merged.

Copilot

Pull request overview

Copilot reviewed 167 out of 177 changed files in this pull request and generated 2 comments.

paulmedynski

Now that we're compiling everything regardless of "test set", I'm not too concerned which test set a test decides to belong to. We're running them all anyway. I feel like we're close to abolishing the test set concept entirely, and separating them into parallelizable chunks based on different criteria anyway.

paulmedynski · 2026-06-08T11:14:52Z

/azp run

azure-pipelines · 2026-06-08T11:15:21Z

Azure Pipelines successfully started running 2 pipeline(s).

Co-authored-by: Paul Medynski <31868385+paulmedynski@users.noreply.github.com>

CI partitions the ManualTests assembly by the "Set" trait. build.proj builds a TestSetFilter of (Set=1|Set=2|Set=3|Set=AE) and ANDs it into the ManualTests filter, and the pipelines run testSets: [1, 2, 3]. IsolationLevelLeakTest was added before the Set trait requirement landed on main (dotnet#4071), so it carried no Set trait. After merging main, the three tests in this file silently stopped matching any CI leg's filter and were never executed -- the PR's own regression tests were producing a false green. Add [Trait("Set", "3")], matching every other class in SQL/TransactionTest (TransactionTest, TransactionEnlistmentTest, DistributedTransactionTest). Verified: - vstest --TestCaseFilter:"Set=3" now lists all three tests; "Set=1" correctly lists none. - All three pass locally against SQL Server 2022 (on-prem, TCP). - No interference when run alongside the other Set=3 transaction and connection-pool tests (19 passed, 1 skipped, 0 failed). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e54f2587-2955-4c93-9896-a35e73feb6e5

Copilot AI review requested due to automatic review settings March 21, 2026 00:53

benrr101 requested a review from a team as a code owner March 21, 2026 00:53

github-project-automation Bot added this to SqlClient Board Mar 21, 2026

github-project-automation Bot moved this to To triage in SqlClient Board Mar 21, 2026

Copilot started reviewing on behalf of benrr101 March 21, 2026 00:54 View session

This comment was marked as outdated.

Sign in to view

paulmedynski moved this from To triage to In review in SqlClient Board Mar 23, 2026

paulmedynski added this to the 7.1.0-preview1 milestone Mar 23, 2026

paulmedynski added the Area\Tests Issues that are targeted to tests or test projects label Mar 23, 2026

paulmedynski requested changes Mar 23, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/AlwaysEncrypted/AKVTests.cs

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/Microsoft.Data.SqlClient.ManualTests.csproj

github-project-automation Bot moved this from In review to In progress in SqlClient Board Mar 23, 2026

paulmedynski self-assigned this Mar 23, 2026

Copilot AI review requested due to automatic review settings March 24, 2026 19:39

Copilot started reviewing on behalf of benrr101 March 24, 2026 19:40 View session

This comment was marked as outdated.

Sign in to view

paulmedynski previously requested changes Mar 25, 2026

View reviewed changes

paulmedynski modified the milestones: 7.1.0-preview1, 7.1.0-preview2 Apr 24, 2026

edwardneal mentioned this pull request Apr 24, 2026

Replace platform-specific builds with runtime OS selection #4239

Open

paulmedynski force-pushed the dev/russellben/cleanup/testsets-categories branch from 892b3de to 85da1f8 Compare May 5, 2026 15:25

paulmedynski moved this from In progress to In review in SqlClient Board May 5, 2026

paulmedynski enabled auto-merge (squash) May 5, 2026 15:28

paulmedynski reviewed May 5, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/Microsoft.Data.SqlClient.ManualTests.csproj Outdated

cheenamalhotra assigned apoorvdeshmukh May 8, 2026

paulmedynski added the Author attention needed PRs that require author to respond or make updates to PR. label May 11, 2026

apoorvdeshmukh reviewed May 14, 2026

View reviewed changes

benrr101 added 2 commits May 22, 2026 13:43

Trait("Set", "..."), and remove manual inclusion of files

f0b29d3

Changes from comments.

f668bbe

paulmedynski requested review from apoorvdeshmukh and paulmedynski June 3, 2026 18:12

paulmedynski previously approved these changes Jun 3, 2026

View reviewed changes

edwardneal reviewed Jun 3, 2026

View reviewed changes

apoorvdeshmukh previously approved these changes Jun 3, 2026

View reviewed changes

paulmedynski added the Author attention needed PRs that require author to respond or make updates to PR. label Jun 4, 2026

Merge branch 'main' into dev/russellben/cleanup/testsets-categories

52b0459

Copilot AI review requested due to automatic review settings June 4, 2026 17:00

benrr101 dismissed stale reviews from apoorvdeshmukh and paulmedynski via 52b0459 June 4, 2026 17:00

benrr101 removed the Author attention needed PRs that require author to respond or make updates to PR. label Jun 4, 2026

Copilot started reviewing on behalf of benrr101 June 4, 2026 17:00 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/VectorTest/NativeVectorFloat32Tests.cs Outdated

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ConnectionPoolTest/ConnectionPoolTest.cs Outdated

paulmedynski previously approved these changes Jun 4, 2026

View reviewed changes

paulmedynski added the Author attention needed PRs that require author to respond or make updates to PR. label Jun 5, 2026

paulmedynski moved this from In review to Waiting for customer in SqlClient Board Jun 5, 2026

Fix traits that belong on test classes.

229711c

benrr101 dismissed paulmedynski’s stale review via 229711c June 5, 2026 19:37

benrr101 removed the Author attention needed PRs that require author to respond or make updates to PR. label Jun 5, 2026

benrr101 moved this from Waiting for customer to In review in SqlClient Board Jun 5, 2026

paulmedynski approved these changes Jun 5, 2026

View reviewed changes

apoorvdeshmukh approved these changes Jun 8, 2026

View reviewed changes

paulmedynski merged commit 1d44a2e into main Jun 8, 2026
301 checks passed

paulmedynski deleted the dev/russellben/cleanup/testsets-categories branch June 8, 2026 17:46

github-project-automation Bot moved this from In review to Done in SqlClient Board Jun 8, 2026

cheenamalhotra pushed a commit that referenced this pull request Jul 14, 2026

Tests | Manual Test "Set" Trait (#4071)

8260e62

Co-authored-by: Paul Medynski <31868385+paulmedynski@users.noreply.github.com>

priyankatiwari08 mentioned this pull request Aug 13, 2026

Reset session isolation level on pool return (fixes #96) #4330

Open

Uh oh!

Conversation

benrr101 commented Mar 21, 2026

Description

🤖

Testing

Uh oh!

This comment was marked as outdated.

Uh oh!

paulmedynski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

paulmedynski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

apoorvdeshmukh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

paulmedynski left a comment

Choose a reason for hiding this comment

Uh oh!

paulmedynski commented Jun 8, 2026

Uh oh!

azure-pipelines Bot commented Jun 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented May 5, 2026 •

edited

Loading