diff --git a/docs/core/project-sdk/msbuild-props.md b/docs/core/project-sdk/msbuild-props.md index 78d550794ece4..553a04cf35a2f 100644 --- a/docs/core/project-sdk/msbuild-props.md +++ b/docs/core/project-sdk/msbuild-props.md @@ -1604,7 +1604,7 @@ For more information, see [Show complete platform output](../testing/unit-testin ### TestingPlatformCommandLineArguments -The `TestingPlatformCaptureOutput` property lets you specify command-line arguments to the test app when you use `dotnet test` to run `Microsoft.Testing.Platform` tests. The following project file snippet shows an example. +The `TestingPlatformCommandLineArguments` property lets you specify command-line arguments to the test app when you use `dotnet test` to run `Microsoft.Testing.Platform` tests. The following project file snippet shows an example. ```xml @@ -1613,6 +1613,8 @@ The `TestingPlatformCaptureOutput` property lets you specify command-line argume ``` +You can also use conditions to pass different arguments to projects that use different test frameworks or extensions. For more information, see [Solutions with mixed test frameworks or extensions](../testing/unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions). + ### TestingPlatformDotnetTestSupport The `TestingPlatformDotnetTestSupport` property enables testing Microsoft.Testing.Platform apps when using the VSTest mode of `dotnet test`. diff --git a/docs/core/testing/microsoft-testing-platform-troubleshooting.md b/docs/core/testing/microsoft-testing-platform-troubleshooting.md index 4e8038c414717..10020e91a8775 100644 --- a/docs/core/testing/microsoft-testing-platform-troubleshooting.md +++ b/docs/core/testing/microsoft-testing-platform-troubleshooting.md @@ -94,3 +94,9 @@ This error can occur if not all of the Fakes assemblies are present in the bin f - Ensure that the project either uses the [MSTest.SDK](./unit-testing-mstest-sdk.md) or references [Microsoft.Testing.Extensions.Fakes](./microsoft-testing-platform-fakes.md). - For .NET Framework projects, avoid setting `AnyCPU` as this results in NuGet not copying all files to the bin folder. + +### Unrecognized command-line option in solutions with mixed test frameworks or extensions + +If your solution contains projects that use different test frameworks (for example, MSTest and xUnit.net) or different sets of extensions (for example, only some projects reference `Microsoft.Testing.Extensions.HangDump`), running `dotnet test` with a framework-specific or extension-specific command-line option can fail with exit code 5. The option is valid for one project but unrecognized by another. + +To resolve this issue, use the `TestingPlatformCommandLineArguments` MSBuild property with conditions to route arguments to the correct projects. For detailed instructions, see [Solutions with mixed test frameworks or extensions](unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions). diff --git a/docs/core/testing/migrating-vstest-microsoft-testing-platform.md b/docs/core/testing/migrating-vstest-microsoft-testing-platform.md index c8b70429931fe..f23600c6eca4f 100644 --- a/docs/core/testing/migrating-vstest-microsoft-testing-platform.md +++ b/docs/core/testing/migrating-vstest-microsoft-testing-platform.md @@ -150,6 +150,9 @@ xUnit.net specific options: For more information, see [Microsoft.Testing.Platform documentation for xUnit.net](https://xunit.net/docs/getting-started/v3/microsoft-testing-platform) and [Query Filter Language for xUnit.net](https://xunit.net/docs/query-filter-language). +> [!TIP] +> If your solution mixes test frameworks that use different filter syntaxes (for example, MSTest and xUnit.net), you can conditionally route framework-specific arguments using the `TestingPlatformCommandLineArguments` MSBuild property. For details, see [Solutions with mixed test frameworks or extensions](unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions). + #### `--logger` What was usually referred to as "logger" in VSTest is referred to as "reporter" in Microsoft.Testing.Platform. In Microsoft.Testing.Platform, logging is explicitly for diagnosing purposes only. diff --git a/docs/core/testing/unit-testing-with-dotnet-test.md b/docs/core/testing/unit-testing-with-dotnet-test.md index 1decbbd4e9289..adbbfdfaa1c0f 100644 --- a/docs/core/testing/unit-testing-with-dotnet-test.md +++ b/docs/core/testing/unit-testing-with-dotnet-test.md @@ -157,3 +157,74 @@ For users of MTP that are using the VSTest mode of `dotnet test`, there are few 1. If passing a specific solution (or directory containing solution), for example, `dotnet test MySolution.sln`, this should become `dotnet test --solution MySolution.sln`. 1. If passing a specific project (or directory containing project), for example, `dotnet test MyProject.csproj`, this should become `dotnet test --project MyProject.csproj`. 1. If passing a specific dll, for example, `dotnet test path/to/UnitTests.dll`, this should become `dotnet test --test-modules path/to/UnitTests.dll`. Note that `--test-modules` also supports globbing. + +## Solutions with mixed test frameworks or extensions + +When a solution contains test projects that use different test frameworks (for example, MSTest and xUnit.net) or different sets of extensions, running `dotnet test` with framework-specific or extension-specific command-line options can fail. Options that are valid for one project are unrecognized by another, causing exit code 5 (invalid command-line arguments). For example: + +- xUnit.net uses `--filter-trait` while MSTest uses `--filter`, and each framework rejects the other's options. +- A project that references `Microsoft.Testing.Extensions.HangDump` accepts `--hangdump`, but a project without that extension fails on the same option. + +Use the [`TestingPlatformCommandLineArguments`](../project-sdk/msbuild-props.md#testingplatformcommandlinearguments) MSBuild property with conditions to pass arguments only to the projects that understand them. + +### Use MSBuild conditions to route arguments + +Define custom MSBuild properties for the framework-specific arguments, and conditionally append them to `TestingPlatformCommandLineArguments` in a `Directory.Build.props` or `Directory.Build.targets` file. Use a condition that checks for a property set by the test framework's SDK or NuGet package to determine which framework each project uses. + +The following example shows a `Directory.Build.props` file for a solution that mixes MSTest (with MSTest.Sdk) and xUnit.net projects: + +```xml + + $(TestingPlatformCommandLineArguments) $(MSTestSpecificArgs) + $(TestingPlatformCommandLineArguments) $(XUnitSpecificArgs) + +``` + +> [!NOTE] +> `UsingMSTestSdk` is a property defined by `MSTest.Sdk`. If your MSTest projects don't use `MSTest.Sdk`, use a different condition. Check whether your test framework's SDK or NuGet package already sets a property you can use. If it doesn't, define your own property in each project and condition on that property instead (in this case, use Directory.Build.targets instead). + +With this configuration in place, you can pass framework-specific arguments from the command line using MSBuild properties: + +```dotnetcli +dotnet test -p:MSTestSpecificArgs="--filter FullyQualifiedName~IntegrationTests" -p:XUnitSpecificArgs="--filter-trait Category=Integration" +``` + +Each test project receives only the arguments relevant to its framework, and the other framework's arguments are never passed. + +### Common scenarios + +The following table lists some common options that differ between frameworks and can be routed using this pattern: + +| Scenario | MSTest / NUnit | xUnit.net | +|----------|---------------|-----------| +| Test filtering | `--filter ` | `--filter-trait`, `--filter-class`, `--filter-method`, `--filter-query` | +| xUnit.net built-in reports | Not available | `--report-xunit-junit`, `--report-xunit-html`, `--report-xunit-trx` | +| Ignore zero tests | `--ignore-exit-code 8` | `--ignore-exit-code 8` | + +> [!TIP] +> For arguments that are the same across all frameworks (such as `--ignore-exit-code 8` or `--report-trx`), set them directly in `TestingPlatformCommandLineArguments` without any condition. + +### Projects with different extensions + +The same pattern applies when only some test projects in a solution reference a particular extension. For example, if only certain projects reference `Microsoft.Testing.Extensions.HangDump`, passing `--hangdump` globally causes the other projects to fail with an unrecognized option error. + +To handle this case, condition the arguments on whether the extension's package is referenced. You can define a helper property in the project files that have the extension, and use that property as a condition: + +```xml + + + $(TestingPlatformCommandLineArguments) $(HangDumpArgs) + +``` + +Then run: + +```dotnetcli +dotnet test -p:HangDumpArgs="--hangdump --hangdump-timeout 10m" +``` + +Only the projects where `EnableHangDump` is `true` receive the hang dump arguments. You can set `EnableHangDump` in each project file that references the extension, or centrally in `Directory.Build.props` with a condition that checks for the package reference. diff --git a/docs/core/tools/dotnet-test-mtp.md b/docs/core/tools/dotnet-test-mtp.md index a83c641f1bbd1..3e39b05a1c6cb 100644 --- a/docs/core/tools/dotnet-test-mtp.md +++ b/docs/core/tools/dotnet-test-mtp.md @@ -163,7 +163,7 @@ With Microsoft Testing Platform, `dotnet test` operates faster than with VSTest. Specifies extra arguments to pass to the test application(s). Use a space to separate multiple arguments. For more information and examples on what to pass, see [Microsoft.Testing.Platform overview](../testing/microsoft-testing-platform-intro.md) and [Microsoft.Testing.Platform features](../testing/microsoft-testing-platform-features.md). > [!TIP] - > To specify extra arguments for specific projects, use the `TestingPlatformCommandLineArguments` MSBuild property. + > To specify extra arguments for specific projects, use the `TestingPlatformCommandLineArguments` MSBuild property. This property is especially useful when your solution mixes test frameworks (for example, MSTest and xUnit.net) or when only some projects reference a particular extension. For more information, see [Solutions with mixed test frameworks or extensions](../testing/unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions). > [!NOTE] > To enable trace logging to a file, use the environment variable `DOTNET_CLI_TEST_TRACEFILE` to provide the path to the trace file.