Skip to content

Fix flaky test: eliminate parallel build conflict in TestDiscoveryWarningsTests#7532

Merged
Youssef1313 merged 3 commits intomainfrom
copilot/fix-potential-test-flakiness
Mar 12, 2026
Merged

Fix flaky test: eliminate parallel build conflict in TestDiscoveryWarningsTests#7532
Youssef1313 merged 3 commits intomainfrom
copilot/fix-potential-test-flakiness

Conversation

Copy link
Contributor

Copilot AI commented Mar 12, 2026

TestDiscoveryWarningsTests was intermittently failing with file-locking errors (IOException: The process cannot access the file ... because it is being used by another process) because TestAssetFixtureBase.InitializeAsync builds all assets in parallel, and TestDiscoveryWarnings has a ProjectReference to TestDiscoveryWarningsBaseClass — so both parallel builds raced to write the same output files.

Changes

  • Embed TestDiscoveryWarningsBaseClass as a subdirectory within the TestDiscoveryWarnings asset instead of yielding it as a separate asset from GetAssetsToGenerate. Only one explicit build is now triggered; MSBuild handles the base class as a project reference dependency.
  • Update ProjectReference from ../TestDiscoveryWarningsBaseClass/... to TestDiscoveryWarningsBaseClass/... to reflect the new relative path.
  • Remove the now-unnecessary BaseTargetAssetPath property and BaseClassSourceCode constant.

Before (two parallel builds → race condition):

yield return (BaseClassAssetName, BaseClassAssetName, BaseClassSourceCode...);
yield return (AssetName, AssetName, SourceCode...);  // ProjectReference: ../TestDiscoveryWarningsBaseClass/...

After (single build, base class as subdirectory):

yield return (AssetName, AssetName, SourceCode...);  // ProjectReference: TestDiscoveryWarningsBaseClass/...
// BaseClass files live at #file TestDiscoveryWarningsBaseClass/TestDiscoveryWarningsBaseClass.csproj

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI linked an issue Mar 12, 2026 that may be closed by this pull request
…to avoid parallel build conflict

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix potential test flakiness in test discovery Fix flaky test: eliminate parallel build conflict in TestDiscoveryWarningsTests Mar 12, 2026
@Evangelink
Copy link
Member

@copilot tests are failing:

test/Utilities/Microsoft.Testing.TestInfrastructure/DotnetCli.cs(176,1): error : [DiscoverTests_ShowsWarningsForTestsThatFailedToDiscover ("net10.0")] [net11.0] Class Initialization method MSTest.Acceptance.IntegrationTests.TestDiscoveryWarningsTests.ClassInitialize threw exception. System.InvalidOperationException: Command 'dotnet build /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings -c Release -p:MSBuildTreatWarningsAsErrors=true -p:TreatWarningsAsErrors=true -p:SuppressNETCoreSdkPreviewMessage=true -bl:"/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/TestDiscoveryWarnings-119.binlog"' failed.

StandardOutput:
Telemetry is: Disabled
  Determining projects to restore...
  Restored /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarningsBaseClass/TestDiscoveryWarningsBaseClass.csproj (in 605 ms).
  Restored /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarnings.csproj (in 749 ms).
  TestDiscoveryWarningsBaseClass -> /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarningsBaseClass/bin/Release/net10.0/TestDiscoveryWarningsBaseClass.dll
/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/UnitTest1.cs(8,27): error CS0436: The type 'BaseClass' in '/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarningsBaseClass/UnitTest1.cs' conflicts with the imported type 'BaseClass' in 'TestDiscoveryWarningsBaseClass, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. Using the type defined in '/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarningsBaseClass/UnitTest1.cs'. [/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarnings.csproj::TargetFramework=net10.0]
Build FAILED.
/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/UnitTest1.cs(8,27): error CS0436: The type 'BaseClass' in '/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarningsBaseClass/UnitTest1.cs' conflicts with the imported type 'BaseClass' in 'TestDiscoveryWarningsBaseClass, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. Using the type defined in '/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarningsBaseClass/UnitTest1.cs'. [/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/kl3sC/TestDiscoveryWarnings/TestDiscoveryWarnings.csproj::TargetFramework=net10.0]
    0 Warning(s)
    1 Error(s)
Time Elapsed 00:00:07.39
StandardError:
.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 12, 2026

@copilot tests are failing:

...

Fixed in 18b9e19. The issue was that MSBuild's default **/*.cs glob in the parent project was picking up TestDiscoveryWarningsBaseClass/UnitTest1.cs, causing the type conflict. Added <Compile Remove="TestDiscoveryWarningsBaseClass/**" /> to exclude the subdirectory's source files from the parent project's compilation.

@Youssef1313 Youssef1313 marked this pull request as ready for review March 12, 2026 09:28
Copilot AI review requested due to automatic review settings March 12, 2026 09:28
@Youssef1313 Youssef1313 enabled auto-merge (squash) March 12, 2026 09:28
@Youssef1313 Youssef1313 disabled auto-merge March 12, 2026 09:29
@Youssef1313 Youssef1313 merged commit 1e46115 into main Mar 12, 2026
4 of 13 checks passed
@Youssef1313 Youssef1313 deleted the copilot/fix-potential-test-flakiness branch March 12, 2026 09:29
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes intermittent TestDiscoveryWarningsTests failures caused by TestAssetFixtureBase.InitializeAsync building multiple generated assets in parallel while one asset had a ProjectReference to another, leading to concurrent MSBuild writes (file locking).

Changes:

  • Stops generating TestDiscoveryWarningsBaseClass as a separate asset; instead embeds it as a subdirectory within the TestDiscoveryWarnings asset so only one explicit build runs.
  • Updates the ProjectReference path to the new in-asset location.
  • Removes the now-unused BaseTargetAssetPath and BaseClassSourceCode constant by folding base-class files into the main SourceCode asset string.

You can also share your feedback on Copilot code review. Take the survey.

<ItemGroup>
<ProjectReference Include="../TestDiscoveryWarningsBaseClass/TestDiscoveryWarningsBaseClass.csproj" />
<ProjectReference Include="TestDiscoveryWarningsBaseClass/TestDiscoveryWarningsBaseClass.csproj" />
<Compile Remove="TestDiscoveryWarningsBaseClass/**" />
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<Compile Remove="TestDiscoveryWarningsBaseClass/**" /> uses a very broad glob that will enumerate all files under that subtree (including bin/obj once they exist) during MSBuild evaluation. Since the intent is only to prevent the main project from compiling the nested base-class sources, restrict the remove pattern to C# sources (e.g., **/*.cs) to avoid unnecessary filesystem scanning and accidental future removals if default items change.

Suggested change
<Compile Remove="TestDiscoveryWarningsBaseClass/**" />
<Compile Remove="TestDiscoveryWarningsBaseClass/**/*.cs" />

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Potential test flakiness

4 participants