Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/core/project-sdk/msbuild-props.md
Original file line number Diff line number Diff line change
Expand Up @@ -1604,7 +1604,7 @@ For more information, see [Show complete platform output](../testing/unit-testin

### TestingPlatformCommandLineArguments

The `TestingPlatformCaptureOutput` property lets you specify command-line arguments to the test app when you use `dotnet test` to run `Microsoft.Testing.Platform` tests. The following project file snippet shows an example.
The `TestingPlatformCommandLineArguments` property lets you specify command-line arguments to the test app when you use `dotnet test` to run `Microsoft.Testing.Platform` tests. The following project file snippet shows an example.

```xml
<PropertyGroup>
Expand All @@ -1613,6 +1613,8 @@ The `TestingPlatformCaptureOutput` property lets you specify command-line argume
</PropertyGroup>
```

You can also use conditions to pass different arguments to projects that use different test frameworks or extensions. For more information, see [Solutions with mixed test frameworks or extensions](../testing/unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions).

### TestingPlatformDotnetTestSupport

The `TestingPlatformDotnetTestSupport` property enables testing Microsoft.Testing.Platform apps when using the VSTest mode of `dotnet test`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,9 @@ This error can occur if not all of the Fakes assemblies are present in the bin f

- Ensure that the project either uses the [MSTest.SDK](./unit-testing-mstest-sdk.md) or references [Microsoft.Testing.Extensions.Fakes](./microsoft-testing-platform-fakes.md).
- For .NET Framework projects, avoid setting `<PlatformTarget>AnyCPU</PlatformTarget>` as this results in NuGet not copying all files to the bin folder.

### Unrecognized command-line option in solutions with mixed test frameworks or extensions

If your solution contains projects that use different test frameworks (for example, MSTest and xUnit.net) or different sets of extensions (for example, only some projects reference `Microsoft.Testing.Extensions.HangDump`), running `dotnet test` with a framework-specific or extension-specific command-line option can fail with exit code 5. The option is valid for one project but unrecognized by another.

To resolve this issue, use the `TestingPlatformCommandLineArguments` MSBuild property with conditions to route arguments to the correct projects. For detailed instructions, see [Solutions with mixed test frameworks or extensions](unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions).
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,9 @@ xUnit.net specific options:

For more information, see [Microsoft.Testing.Platform documentation for xUnit.net](https://xunit.net/docs/getting-started/v3/microsoft-testing-platform) and [Query Filter Language for xUnit.net](https://xunit.net/docs/query-filter-language).

> [!TIP]
> If your solution mixes test frameworks that use different filter syntaxes (for example, MSTest and xUnit.net), you can conditionally route framework-specific arguments using the `TestingPlatformCommandLineArguments` MSBuild property. For details, see [Solutions with mixed test frameworks or extensions](unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions).

#### `--logger`

What was usually referred to as "logger" in VSTest is referred to as "reporter" in Microsoft.Testing.Platform. In Microsoft.Testing.Platform, logging is explicitly for diagnosing purposes only.
Expand Down
71 changes: 71 additions & 0 deletions docs/core/testing/unit-testing-with-dotnet-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,3 +157,74 @@ For users of MTP that are using the VSTest mode of `dotnet test`, there are few
1. If passing a specific solution (or directory containing solution), for example, `dotnet test MySolution.sln`, this should become `dotnet test --solution MySolution.sln`.
1. If passing a specific project (or directory containing project), for example, `dotnet test MyProject.csproj`, this should become `dotnet test --project MyProject.csproj`.
1. If passing a specific dll, for example, `dotnet test path/to/UnitTests.dll`, this should become `dotnet test --test-modules path/to/UnitTests.dll`. Note that `--test-modules` also supports globbing.

## Solutions with mixed test frameworks or extensions

When a solution contains test projects that use different test frameworks (for example, MSTest and xUnit.net) or different sets of extensions, running `dotnet test` with framework-specific or extension-specific command-line options can fail. Options that are valid for one project are unrecognized by another, causing exit code 5 (invalid command-line arguments). For example:

- xUnit.net uses `--filter-trait` while MSTest uses `--filter`, and each framework rejects the other's options.
- A project that references `Microsoft.Testing.Extensions.HangDump` accepts `--hangdump`, but a project without that extension fails on the same option.

Use the [`TestingPlatformCommandLineArguments`](../project-sdk/msbuild-props.md#testingplatformcommandlinearguments) MSBuild property with conditions to pass arguments only to the projects that understand them.

### Use MSBuild conditions to route arguments

Define custom MSBuild properties for the framework-specific arguments, and conditionally append them to `TestingPlatformCommandLineArguments` in a `Directory.Build.props` or `Directory.Build.targets` file. Use a condition that checks for a property set by the test framework's SDK or NuGet package to determine which framework each project uses.

The following example shows a `Directory.Build.props` file for a solution that mixes MSTest (with MSTest.Sdk) and xUnit.net projects:

Comment thread
Evangelink marked this conversation as resolved.
```xml
<PropertyGroup>
<TestingPlatformCommandLineArguments
Condition="'$(UsingMSTestSdk)' == 'true'"
>$(TestingPlatformCommandLineArguments) $(MSTestSpecificArgs)</TestingPlatformCommandLineArguments>
<TestingPlatformCommandLineArguments
Condition="'$(UsingMSTestSdk)' != 'true'"
>$(TestingPlatformCommandLineArguments) $(XUnitSpecificArgs)</TestingPlatformCommandLineArguments>
</PropertyGroup>
```

> [!NOTE]
> `UsingMSTestSdk` is a property defined by `MSTest.Sdk`. If your MSTest projects don't use `MSTest.Sdk`, use a different condition. Check whether your test framework's SDK or NuGet package already sets a property you can use. If it doesn't, define your own property in each project and condition on that property instead (in this case, use Directory.Build.targets instead).

With this configuration in place, you can pass framework-specific arguments from the command line using MSBuild properties:

```dotnetcli
dotnet test -p:MSTestSpecificArgs="--filter FullyQualifiedName~IntegrationTests" -p:XUnitSpecificArgs="--filter-trait Category=Integration"
```

Each test project receives only the arguments relevant to its framework, and the other framework's arguments are never passed.

### Common scenarios
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole "Common scenarios" section is too wordy IMO without giving user and more useful info. Everything seems to be well explained from previous.


The following table lists some common options that differ between frameworks and can be routed using this pattern:

| Scenario | MSTest / NUnit | xUnit.net |
|----------|---------------|-----------|
| Test filtering | `--filter <expression>` | `--filter-trait`, `--filter-class`, `--filter-method`, `--filter-query` |
| xUnit.net built-in reports | Not available | `--report-xunit-junit`, `--report-xunit-html`, `--report-xunit-trx` |
| Ignore zero tests | `--ignore-exit-code 8` | `--ignore-exit-code 8` |
Comment thread
Evangelink marked this conversation as resolved.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get the point you're trying to make by this example.


> [!TIP]
> For arguments that are the same across all frameworks (such as `--ignore-exit-code 8` or `--report-trx`), set them directly in `TestingPlatformCommandLineArguments` without any condition.

### Projects with different extensions

The same pattern applies when only some test projects in a solution reference a particular extension. For example, if only certain projects reference `Microsoft.Testing.Extensions.HangDump`, passing `--hangdump` globally causes the other projects to fail with an unrecognized option error.

To handle this case, condition the arguments on whether the extension's package is referenced. You can define a helper property in the project files that have the extension, and use that property as a condition:

```xml
<!-- In the .csproj files that reference HangDump (or via Directory.Build.targets) -->
<PropertyGroup Condition="'$(EnableHangDump)' == 'true'">
<TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) $(HangDumpArgs)</TestingPlatformCommandLineArguments>
</PropertyGroup>
```

Then run:

```dotnetcli
dotnet test -p:HangDumpArgs="--hangdump --hangdump-timeout 10m"
```

Only the projects where `EnableHangDump` is `true` receive the hang dump arguments. You can set `EnableHangDump` in each project file that references the extension, or centrally in `Directory.Build.props` with a condition that checks for the package reference.
2 changes: 1 addition & 1 deletion docs/core/tools/dotnet-test-mtp.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ With Microsoft Testing Platform, `dotnet test` operates faster than with VSTest.
Specifies extra arguments to pass to the test application(s). Use a space to separate multiple arguments. For more information and examples on what to pass, see [Microsoft.Testing.Platform overview](../testing/microsoft-testing-platform-intro.md) and [Microsoft.Testing.Platform features](../testing/microsoft-testing-platform-features.md).

> [!TIP]
> To specify extra arguments for specific projects, use the `TestingPlatformCommandLineArguments` MSBuild property.
> To specify extra arguments for specific projects, use the `TestingPlatformCommandLineArguments` MSBuild property. This property is especially useful when your solution mixes test frameworks (for example, MSTest and xUnit.net) or when only some projects reference a particular extension. For more information, see [Solutions with mixed test frameworks or extensions](../testing/unit-testing-with-dotnet-test.md#solutions-with-mixed-test-frameworks-or-extensions).

> [!NOTE]
> To enable trace logging to a file, use the environment variable `DOTNET_CLI_TEST_TRACEFILE` to provide the path to the trace file.
Expand Down
Loading