Skip to content

Evaluate VS Test Platform as a better replacement for test discovery and execution#83

Open
janaka wants to merge 15 commits intomasterfrom
janaka-vstestplatform-integartion-take2
Open

Evaluate VS Test Platform as a better replacement for test discovery and execution#83
janaka wants to merge 15 commits intomasterfrom
janaka-vstestplatform-integartion-take2

Conversation

@janaka
Copy link
Copy Markdown
Collaborator

@janaka janaka commented May 24, 2018

The purpose of this PR is to evaluate VS Test Platform in more depth, understand if it solves the below problems, and hence if we should move over.

Currently we use the dotnet test and dotnet vstest for test execution and generate TRX files which we parse for test discovery. This approach has several limitations.

  • Doesn't support multiple test projects per workspace.
  • Doesn't scale well to 100s of tests. We theorise performance will suffer.
  • Differing output formats between test frameworks complicates code and testing; has resulted in more bugs; and inability to support some functionality consistently across frameworks.

VS Test Platform is the underlying system, which has been open sourced, that dotnet vstest commands and Visual Studio test explorer is build upon. My understanding is that it has a superset of functionality compared to the dotnet cli and designed to enable scalable and performant test discovery and execution tools.

Note that VS Test Platform project itself doesn't have any client code, it's a backend + remote API. Hence this is also building on the great work over at https://github.com/gfrancischini/vscode-vstest-framework

Evaluation tasks are here

If any are missing please comment below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant