Solution wide coverage #357
Comments
|
FYI, The |
|
That'll produce individual test reports, but won't produce a single combined test report (my impression of what this enhancement is for). |
|
@IGx89 if you run it will create a single coverage file and will report the complete coverage after each test. some things to consider with this approach:
|
|
Directory.Build.props: <PropertyGroup>
<CoverletOutput>./coverage/coverage.xml</CoverletOutput>
<CoverletOutputFormat>opencover</CoverletOutputFormat>
</PropertyGroup>Directory.Build.targets: <Target Name="CoverRunPreparation" BeforeTargets="GenerateCoverageResult" Condition="'$(CollectCoverage)' == 'true'">
<ItemGroup>
<CoverletOutputPath Include="$(CoverletOutput)" />
</ItemGroup>
<PropertyGroup>
<CoverletOutput Condition="'$(CoverletOutput)' != '' AND '$(TargetFrameworks)' != ''">%(CoverletOutputPath.RootDir)/%(CoverletOutputPath.Directory)/%(CoverletOutputPath.Filename).$(TargetFramework)%(CoverletOutputPath.Extension)</CoverletOutput>
<CoverletOutput Condition="'$(CoverletOutput)' == '' AND '$(TargetFrameworks)' == ''">$(ProjectDir)coverage.xml</CoverletOutput>
<CoverletOutput Condition="'$(CoverletOutput)' == '' AND '$(TargetFrameworks)' != ''">$(ProjectDir)coverage.$(TargetFramework).xml</CoverletOutput>
</PropertyGroup>
</Target> |
|
@joemey how does |
@iron9light take a look at #474 |
MarcoRossignoli
added
feature-request
|
I have tried the solution suggested by @joemey and it works, but only for For this command So it counts coverage for the first project and saves it into Is it possible to count coverage and merge into one |
|
@lkurzyniec can you try to run same command but with
Yes I did same sample on our "sample" section https://github.com/tonerdo/coverlet/blob/master/Documentation/Examples/MSBuild/MergeWith/HowTo.md , you can clone repo and test solution, but you cannot run command "solution wide". |
|
For reference link msbuild task sync discussion dotnet/msbuild#4898 dotnet/msbuild#4954 |
|
Unfortunately, it doesn't work What's interesting here is that it generate json two times: |
|
Try with Similar to https://stackoverflow.com/questions/53255065/dotnet-unit-test-with-coverlet-how-to-get-coverage-for-entire-solution-and-not/56281108#56281108 but with For instance you can try with our samples on repo |
|
@MarcoRossignoli That works fine! In my case, it gives me solution wide results. Summarycmd command dotnet test /p:CollectCoverage=true /p:CoverletOutput="../" /p:MergeWith="../coverage.json" /maxcpucount:1 /p:CoverletOutputFormat=\"json,opencover\"gives me the same results as following two commands dotnet test ./test/HappyCode.NetCoreBoilerplate.Core.UnitTests/HappyCode.NetCoreBoilerplate.Core.UnitTests.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/
dotnet test ./test/HappyCode.NetCoreBoilerplate.Api.UnitTests/HappyCode.NetCoreBoilerplate.Api.UnitTests.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat="opencover"
What's more, your command |
|
Glad to hear! |
|
Not sure if this should be a separate question, or asked here, but is it possible to do solution wide coverage using the coverlet .Net global tool? I'm using coverlet in Azure Pipelines and would like it to be system agnostic (Linux, Windows, Mac). The task I'm running looks like this: But could it target a directory and run all tests projects under that directory? Or target a solution file? |
Why don't use collectors https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md? Why .NET tool? Coverlet is cross platform and works cross plat for every "drivers" you're using https://github.com/tonerdo/coverlet#quick-start but the collectors is the preferred way to consume due to integration with vstest platform and you can run command on sln(it generates one coverage file for every test project you have in solution). |
|
At this stage I was only using coverlet as part of the Azure Pipeline, so didn't think the test projects should know or depend on it. Will try installing coverlet.collector as a NuGet package. |
|
So I added coverlet.collector as a NuGet package and got it going for a single test project with this: But this produces a separate coverage.cobertura.xml file per test project (each in there own guid directory). In this Azure Pipelines context how would I get the individual test results merged together so that a single code coverage publish can be done? |
|
@robertlarkins my suggestion would be to read both this article and whole this thread. Then try to achieve solution-wide results locally. After that move your solution to Azure DevOps. |
|
@lkurzyniec Will attempt and see how I get on. |
|
At the moment I've removed from documentation Actually that parameters is "hidden" but usable, so if you want be sure to not hit https://github.com/tonerdo/coverlet/blob/master/Documentation/KnowIssues.md#1-vstest-stops-process-execution-earlydotnet-test and use merge with vstest you can use a trick of our contributor @p4p3 that with a script move generated coverage files for every test run #225 (comment) If you don't hit issue above you can use msbuild driver and https://github.com/tonerdo/coverlet/blob/master/Documentation/Examples/MSBuild/MergeWith/HowTo.md |
|
@lkurzyniec , @MarcoRossignoli Hi. I am trying to execute the above command but getting an error |
|
@sungam3r you need to translate characters on powershell take a look https://github.com/tonerdo/coverlet/blob/master/Documentation/MSBuildIntegration.md#note-for-powershell--vsts-users |
|
Wow! Thanks. I read this section of the documentation, but for some reason I missed it. |
|
No prob |
|
Still having issues in getting merge results and allowing to run ReportGenerator from that. Following is the
|
|
And I have tried different variations of this CoverletOutputFormat string, all resulting to error of some sort with command above: |
|
And found my problem, so I am not supposed to put the json file, but only a path to |
|
@aviita sorry for the delay(very busy time), glad to hear you solve your problem. |
|
It was helpfull for me. I use coverall and special solutions for unit and functional tests. |
|
Solution-wide coverage reports seem to work well using dotnet test /p:CollectCoverage=true /p:CoverletOutput="../CoverageResults/" /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat="\"cobertura,json\"" /p:Threshold=$MIN_CODE_COVERAGE_PERCENTAGE /p:ThresholdType=line /p:ThresholdStat=totalThe solution-wide reports seem to be okay, but the threshold is checked after each test project is run, on the total coverage so far. Depending on the order test projects are run in this can cause our CI pipeline to fail even if overall coverage is actually above the threshold. I found a workaround that involves running tests multiple times, but that's kind of hacky and adds unnecessary run time. Is there a better supported approach to accurate solution-wide thresholds? #598 seems relevant, but it was closed as a duplicate of this one so I'm asking here. Thanks in advance for any advice, and thanks for your work maintaining coverlet. |
|
FWIW, my team only need thresholds at a project level (#1227) but wish to report coverage at a solution level. Hopefully it should be obvious that test projects should not be included in threshold checks nor in reporting. Setting thresholds, inclusions/exclusions, etc at a solution level is going to make for one very big command line though. |
Coverlet should be able to get coverage for all test projects in a specified solution file
related issues
#165
The text was updated successfully, but these errors were encountered: