Empty cobertura.xml coverage files

[AArnott]

My TUnit test project targets 4 frameworks (net472, net8.0, net9.0, net10.0). In a test run, each produce a cobertura.xml file, but only one is non-empty.

https://github.com/AArnott/Nerdbank.MessagePack/actions/runs/20447346767/job/58753319027?pr=834#step:6:788

The 3 that are empty all look like this:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<coverage line-rate="1" branch-rate="1" complexity="1" version="1.9" timestamp="1766448936">
  <packages />
</coverage>

That's the whole file.

I believe this underreporting is why in my xunit to tunit migration, I've reportedly lost about 5% code coverage.

Do you know why this is happening?

When I run tests within VS with code coverage enabled, the coverage numbers are... different. But they must somehow be counted quite differently. The product code (which doesn't change at all across the xunit-to-tunit migration) allegedly is made up of a different number of code blocks.
Here is the in-IDE coverage numbers of a product class before and after the migration (both as seen on a .NET 9 test run):

Comparison	Xunit	TUnit
Covered (Blocks)	559	737
Not Covered (Blocks)	92	121
Covered (Lines)	362	483
Partially Covered (Lines)	7	12
Not Covered (Lines)	45	64

These don't add up to the same numbers when you add covered and non-covered things.

[SeanGriffin-Wellsky]

In your XUnit version are you using the mtp-v2 version?
Are you using Microsoft.Testing.Extensions.CodeCoverage as your code coverage tool?
Do you get different behavior when running dotnet test from CLI vs. Visual Studio?

I have done a similar XUnit v3 MTP 2 migration to TUnit to compare the results, using Microsoft.Testing.Extensions.CodeCoverage as my package, and I do not see a difference in the reported covered lines or branches. This is when ran via CLI and CI pipeline. That said, in my case I'm dual targeting just net8 and net10.

[JohnVerheij]

We hit this exact signature this week and confirmed the trigger. Posting in case useful for narrowing your case.

Reproduction

• Project: TUnit-based test suite (around 2,600 TUnit tests across multiple sub-projects); the discussion below tracks one Verify.TUnit-using sub-project that emits its own per-project cobertura.
• Environment: Debian trixie-slim container, .NET SDK 10.0.203, Microsoft.Testing.Extensions.CodeCoverage 18.6.2.
• Local Docker on this image: 769 KB cobertura for that sub-project, populated with class and line entries.
• GitLab CI Linux runner on the same image, same packages, same shell session: 178-byte degenerate <coverage line-rate="1"...><packages /></coverage> for that sub-project, while the 4 sibling sub-projects in the same CI invocation produce normal cobertura. Tests pass in both runs.

Trigger

The failing sub-project referenced Verify.TUnit. Verify.props (lines 4-5) sets:

<Deterministic>false</Deterministic>
<DeterministicSourcePaths>false</DeterministicSourcePaths>

for any project that pulls Verify in. The MSBuild binary log confirms this empirically: Deterministic = false evaluates in the failing project; Deterministic = true evaluates in each of the four working sibling projects (none reference Verify). With this setting active on the GitLab runner environment, Microsoft.Testing.Extensions.CodeCoverage produces the degenerate cobertura.

Confirming experiment

Adding <Deterministic>true</Deterministic> to the Verify-using csproj (which overrides Verify's setting via post-import MSBuild evaluation order) produces real per-project cobertura on the same CI environment: 774 KB populated with class and line entries instead of 178 bytes. Only that property changed between the two CI runs.

Design tension caveat

The override is not a usable fix. Verify needs the non-deterministic absolute source paths in PDBs to locate .verified.txt files at runtime. With Deterministic=true, paths are normalised in the PDB to the /_/... form, and the test process writes received files at paths like /_/tests/<project>/<TestClass>.<TestMethod>.received.txt, which do not exist on disk. Verify cannot find the corresponding .verified.txt and snapshot tests fail.

In our case we have accepted the trade-off of leaving snapshot tests out of coverage measurement (our snapshot tests verify serialised output rather than code-path behaviour, so the lost signal is bounded for us). Whether that trade-off is acceptable for any given project depends on what its snapshot tests are checking.

Worth checking for your case

Does your project (or any sub-project) reference Verify? If yes, the same Deterministic=false mechanism may be at play. If you do not reference Verify, the trigger is something else and our finding is just adjacent.

Caveat we have not explained

We do not yet understand why local Docker on the exact same image does not reproduce the empty cobertura. Same SDK, same package versions, same coverage settings, same Deterministic=false. Something about the GitLab runner environment combines with Deterministic=false to trip the failure. Open question.

[thomhurst]

Sorry I never saw this!

The code coverage package and functionality is supplied by Microsoft, so unfortunately I have the rubbish answer of I don't know why it's different! Should be able to ask here though: https://github.com/microsoft/codecoverage