TUnit Framework

[DavidPotter01]

TUnit is a newer testing framework that utilizes Microsoft's new testing package Microsoft.Testing.Platform. After installing FCC extension and running a TUnit test, I get the messsage "No cobertura files for ms code coverage." What am I missing to get FCC to pick up these tests and show the code coverage in the FCC window?

[DavidPotter01]

I have RunMsCodeCoverage set to Yes and IncludeReferencedProjects set to True

[tonyhallett]

Are you using a test environment ?

Can you disable the vs feature - Options / Environment / Preview Features / Use testing platform server mode"

Can you provide a miminal solution so that I can debug FCC against it ?

[tonyhallett]

Can you also follow the instructions https://github.com/microsoft/VSProjectSystem/blob/master/doc/overview/about_project_capabilities.md#viewing-a-projects-capabilities to determine the test project capabilities.

[tonyhallett]

On the TUnit readme
https://github.com/microsoft/VSProjectSystem/blob/master/doc/overview/about_project_capabilities.md#viewing-a-projects-capabilities

If for TUnit does require the setting then it is not supported.

Perhaps in Vs version 17.13 it is not possible for FCC to do it's switching.

What version of VS are you using ?

[DavidPotter01]

I am on the latest so 17.13. So FineCodeCoverage cannot be used with latest version of visual studio since that preview setting is now a permanently enabled setting in VS? Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Tony Hallett ***@***.***> Sent: Saturday, March 15, 2025 4:39:35 AM To: FortuneN/FineCodeCoverage ***@***.***> Cc: David Potter ***@***.***>; Author ***@***.***> Subject: Re: [FortuneN/FineCodeCoverage] TUnit Framework (Discussion #497) On the TUnit readme https://github.com/microsoft/VSProjectSystem/blob/master/doc/overview/about_project_capabilities.md#viewing-a-projects-capabilities If for TUnit does require the setting then it is not supported. Perhaps in Vs version 17.13 it is not possible for FCC to do it's switching. What version of VS are you using ? — Reply to this email directly, view it on GitHub<#497 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BLZXKX3EQG7R6JV3HHH2KH32UPRMPAVCNFSM6AAAAABZCAEBWWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENJQHAZDGMI>. You are receiving this because you authored the thread.Message ID: ***@***.***> CONFIDENTIAL The information contained in this electronic message is confidential and intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this electronic message is strictly prohibited. If you have received this electronic message in error, please notify me immediately at the above listed telephone number(s) and delete the original message.

[tonyhallett]

Possibly not. I would have to check the feature settings, source code and msbuild.

Can you show your project capabilities.

If you can also provide a minimal solution with TUnit to save me 5 minutes.

I will look at Monday.

[tonyhallett]

FCC coverage works for TUnit in 17.12.3.
The msbuild property we change to remove the testing platform server is still present ( although it is in a different file from before ) https://github.com/microsoft/testfx/blob/eba5fbbe867519fe2304ec606e76bc2e6d0eefa5/src/Platform/Microsoft.Testing.Platform/buildMultiTargeting/Microsoft.Testing.Platform.targets#L12
FCC currently turns off conditionally

            if (await UsingTestingPlatformProtocolAsync() && !AllProjectsDisabled() && await ProjectEnabledAsync())
            {
                return Empty.PropertiesMap.Add("DisableTestingPlatformServerCapability", "true");
            }

We may be able to use remove UsingTestingPlatformProtocolAsync()

I am updating my vs version now. Will let you know.

[tonyhallett]

Take my earlier comment of it working in the previous version as I believe that I had not set UseMsCodeCoverage to Yes.

I think that MsTest with the testing platform works because it has the test adapter present. Whereas TUnit does not have one.
TUnit cannot be supported with the way that FCC currently collects coverage using the Visual Studio test explorer.

I will investigate what the options are.

[tonyhallett]

XUnit, NUnit and MSTest ( MSTest requires a change in FCC code ) all work with RunMsCodeCoverage Yes with Visual Studio 17.13.3
You can get code coverage for TUnit with RunMsCodeCoverage No although this does mean that tests are run twice and FCC copies bin/debug to another directory.

For TUnit FCC is unable to use the test explorer window as there is no fallback available and it is not possible to use https://www.nuget.org/packages/Microsoft.Testing.Extensions.CodeCoverage for coverage as the test window controls the arguments and that is the only way that Microsoft.Testing.Extensions.CodeCoverage can be configured. I have opened a feature request as it could provide a hook for FCC.

The only other solutions are for FCC to have a buttons

Import a cobertura file that you generate but then you would need to provide the settings
FCC runs and collects the coverage with dotnet test etc - which is the better option

[DavidPotter01]

I was going to mention that I can generate .coverage files using Microsoft.Testing.Extensions.CodeCoverage by executing dotnet through the command-line (either dotnet test or dotnet run) with --coverage but I see the testing window does not support passing in arguments.

I appreciate you looking into this, what would you like to do moving forward? Have a temp solution that someone can do in the meantime while we wait of the hook to allow you to pass in arguments?
A button would be a simple user experience in the meantime but not going to be picky.

[DavidPotter01]

I set RunMsCodeCoverage to No and I do see FCC rerun and can see the FCC view populate, but it says code coverage is 0% where I can see that code is being covered for class where the test passes. Even says the Test project methods have 0% coverage

[tonyhallett]

Apologies, yes that is correct. Forget that idea for the moment then.

[tonyhallett]

what would you like to do moving forward? Have a temp solution that someone can do in the meantime while we wait of the hook to allow you to pass in arguments?
A button would be a simple user experience in the meantime but not going to be picky.

There are two hacky solutions I can think of
I. If the test explorer starts the exe with Process, FCC could provide an MsBuild target to wrap the exe to provide the missing arguments
2. Replace https://www.nuget.org/packages/Microsoft.Testing.Extensions.CodeCoverage with an FCCCodeCoverage extension that invokes the Microsoft.Testing.Extensions.CodeCoverage TestingPlatformBuilderHook with wrapper types so that when its builder code invokes IServerProvider.GetCommandLineOptions a wrapped ICommandLineOptions is returned.

and perhaps not so hacky
The testing platform build process creates two files that you can see in obj/Debug

AutoRegisteredExtension.cs

	//------------------------------------------------------------------------------
	// <auto-generated>
	//     This code was generated by Microsoft.Testing.Platform.MSBuild
	// </auto-generated>
	//------------------------------------------------------------------------------
	
	[global::System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage]
	internal static class SelfRegisteredExtensions
	{
	    public static void AddSelfRegisteredExtensions(this global::Microsoft.Testing.Platform.Builder.ITestApplicationBuilder builder, string[] args)
	    {
	        Microsoft.Testing.Platform.MSBuild.TestingPlatformBuilderHook.AddExtensions(builder, args);
	        TUnit.Engine.Framework.TestingPlatformBuilderHook.AddExtensions(builder, args);
	    }
}

and more importantly
TestPlatformEntryPoint

	//------------------------------------------------------------------------------
	// <auto-generated>
	//     This code was generated by Microsoft.Testing.Platform.MSBuild
	// </auto-generated>
	//------------------------------------------------------------------------------
	
	[global::System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage]
	internal sealed class TestingPlatformEntryPoint
	{
	    public static async global::System.Threading.Tasks.Task<int> Main(string[] args)
	    {
	        global::Microsoft.Testing.Platform.Builder.ITestApplicationBuilder builder = await global::Microsoft.Testing.Platform.Builder.TestApplication.CreateBuilderAsync(args);
	        SelfRegisteredExtensions.AddSelfRegisteredExtensions(builder, args);
	        using (global::Microsoft.Testing.Platform.Builder.ITestApplication app = await builder.BuildAsync())
	        {
	            return await app.RunAsync();
	        }
	    }
}

FCC could add code to add additional args.

Of course for any of these methods FCC needs to be ignored when not running from test explorer.

I will give some thought to the buttons. If it is just a single project TUnit test project or all TUnit then that is probably fine, but a mixture of test projects types would not work well.

[DavidPotter01]

I would think that anyone running tests for their project would use the same testing framework for all their test projects inside a solution.

[tonyhallett]

Agreed, but there is always one

[tonyhallett]

I have implemented a button that builds the test projects and all dependencies in debug mode, then executes the exe with --coverage etc.

This is a branch off the main branch so the FCC window looks different. ( The report is WPF instead of the current html - still has work to be done but you get the idea )

I still have the coverage settings to do. Button image tbd.

Once I am happy I'll send you a link to vsix that you can try before I release. I'll also provide details of the source as you may have thoughts on the build process.

[DavidPotter01]

I updated to latest FCC and I'm getting my unit tests not loading anymore. I also notice the icons for my test project are flickering between C# and the Test icon constantly.

[tonyhallett]

Is this in a TUnit test project ?

[tonyhallett]

If so, the current code reads the project file for a TUnit package reference and if it finds it does not do the switching. FCC can do a more thorough check.

If this is TUnit disabling FCC should prevent this behaviour

[tonyhallett]

Latest release has a button on the toolbar for TUnit tests and your tests not loading issue should be fixed. You can pass command line args using msbuild properties, please see the readme. Any issues let me know.

[DavidPotter01]

I see the button, but I am still encountering the flickering issue on the projects and still get the same error message when I enable FCC (after updating FCC).
It looks like it may be bacause I have 1 base Test project that all my other Test projects reference. The base Test project has the TUnit reference while all the other Test projects do not, they just reference the base Test project.

I got the flickering to go away if I explicitly add TUnit to each and every project (as a workaround right now). Although I do not get the loading failure, but when click the "Collect TUnit" button, I do not get any results and see this in the FCC logs

[tonyhallett]

Are you using FCC 1.1.242 ?
How many TUnit test projects do you have ?

[tonyhallett]

Does base produce a testing platform exe ?

[DavidPotter01]

it's a console app

[DavidPotter01]

though it might make more sense it should be a library project since no TUnit tests are being executed inside it (TUnit needs to have the project be a Console App for the tests to run).

[tonyhallett]

If it is not intended to be executable and has no tests then I think it should be a library. Then you import the TUnit packages that you need. Do this and there will be no flickering and you will receive coverage from FCC.

TUnit.Core
To be used when you want to define re-useable components, such as a test library, but it wouldn't be run as its own test suite.

TUnit.Engine
For test suites. This contains the test execution logic and test adapter. Only install this on actual test projects you intend to run, not class libraries.

( It also contains the TestingPlatformBuilderHook MsBuild prop for TUnit to declare itself as a testing framework so that it can participate in the building process )

[DavidPotter01]

yep that seems to have solved it. Since I am not using Coverlet, how do I exclude all these test projects since they are being included in the code coverage?

[tonyhallett]

FCC options - Exclude / Include ( Common ) - IncludeTestAssembly false.

[tonyhallett]

BaseTests is probably not covered by this.
Apply System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute to the assembly.
Or Exclude / Include ( Microsoft ) - add a ModulePathsInclude regex.

[tonyhallett]

The TUnit template normally excludes test projects.

https://github.com/thomhurst/TUnit/blob/7ed0ecfa9dc5b632c1a17eac54d1b337ffa83584/TUnit.Templates/content/TUnit/GlobalSetup.cs#L4