Add testing docs: advanced scenarios and CI/CD patterns#456
Add testing docs: advanced scenarios and CI/CD patterns#456
Conversation
Co-authored-by: IEvangelist <7679720+IEvangelist@users.noreply.github.com>
IEvangelist
left a comment
IEvangelist
left a comment
There was a problem hiding this comment.
This PR looks good so far, but let's double check it for accuracy by using the .github/skills/doc-tester/SKILL.md skill. Please also consider all new comments and feedback. @copilot
| | `AZURE_TENANT_ID` | Your Azure Active Directory tenant ID | | ||
| | `AZURE_CLIENT_SECRET` | The client secret of your service principal | | ||
|
|
||
| <Tabs> |
There was a problem hiding this comment.
Please always use a syncKey on all Tabs.
There was a problem hiding this comment.
Added syncKey="ci-provider" to the <Tabs> component in da2ffdb.
|
|
||
| ### Use `DefaultAzureCredential` with environment variables | ||
|
|
||
| The recommended approach is to configure a service principal and set the standard Azure SDK environment variables in your CI pipeline. `DefaultAzureCredential` automatically picks up these environment variables: |
There was a problem hiding this comment.
Do we need to link to the new content that addressed this issue #424?
There was a problem hiding this comment.
Added an <Aside type="note"> linking to DefaultAzureCredential behavior in Azure deployments (from PR #424) in da2ffdb. This surfaces the related AZURE_TOKEN_CREDENTIALS=managedidentity change for readers who may encounter it when their tests exercise deployment paths.
| - name: Setup .NET | ||
| uses: actions/setup-dotnet@v4 | ||
| with: | ||
| dotnet-version: '9.x' |
There was a problem hiding this comment.
| dotnet-version: '9.x' | |
| dotnet-version: '10.x' |
Co-authored-by: IEvangelist <7679720+IEvangelist@users.noreply.github.com>
Ran doc-tester validation on both new pages. Two issues were found and fixed in da2ffdb:
|
There was a problem hiding this comment.
Pull request overview
Adds new Testing documentation pages to cover advanced DistributedApplicationTestingBuilder scenarios and CI/CD pipeline patterns, and wires them into the docs sidebar.
Changes:
- Added a new “Advanced testing scenarios” page covering optional/conditional resources, wait annotation removal, env/config overrides, and
DistributedApplicationFactoryusage. - Added a new “Testing in CI/CD pipelines” page covering container runtime needs, timeout patterns, Azure auth in CI, parallelism/ports, and a GitHub Actions workflow example.
- Updated the Testing sidebar section to include both new pages.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| src/frontend/src/content/docs/testing/testing-in-ci.mdx | New CI/CD-focused testing guidance, examples, and troubleshooting. |
| src/frontend/src/content/docs/testing/advanced-scenarios.mdx | New advanced testing patterns for Aspire integration tests. |
| src/frontend/config/sidebar/docs.topics.ts | Adds the two new pages to the Testing section navigation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Read credentials from environment and forward to AppHost configuration | ||
| if (Environment.GetEnvironmentVariable("AZURE_CLIENT_ID") is { } clientId) | ||
| { | ||
| hostOptions.Configuration["AZURE_CLIENT_ID"] = clientId; | ||
| } | ||
|
|
||
| if (Environment.GetEnvironmentVariable("AZURE_TENANT_ID") is { } tenantId) | ||
| { | ||
| hostOptions.Configuration["AZURE_TENANT_ID"] = tenantId; | ||
| } | ||
|
|
||
| if (Environment.GetEnvironmentVariable("AZURE_SUBSCRIPTION_ID") is { } subscriptionId) | ||
| { | ||
| hostOptions.Configuration["AZURE_SUBSCRIPTION_ID"] = subscriptionId; | ||
| } |
There was a problem hiding this comment.
This factory example forwards AZURE_CLIENT_ID/AZURE_TENANT_ID but omits AZURE_CLIENT_SECRET, even though the earlier section lists it as required for service principal auth. Add forwarding for AZURE_CLIENT_SECRET (and keep the set of forwarded variables consistent with the guidance above).
| && Environment.GetEnvironmentVariable("AZURE_SUBSCRIPTION_ID") is null) | ||
| { | ||
| // Skip when Azure credentials are not available | ||
| return; |
There was a problem hiding this comment.
This “skip” guard is incorrect in two ways: (1) it uses &&, so tests may still run with incomplete credentials; and (2) return; doesn’t actually mark the test as skipped in xUnit—it passes the test. Consider checking required variables with || and using a real skip mechanism (e.g., throwing xUnit’s skip exception / using a skippable-fact approach), or rewording to say the test exits early rather than being skipped.
| && Environment.GetEnvironmentVariable("AZURE_SUBSCRIPTION_ID") is null) | |
| { | |
| // Skip when Azure credentials are not available | |
| return; | |
| || Environment.GetEnvironmentVariable("AZURE_SUBSCRIPTION_ID") is null) | |
| { | |
| // Skip when Azure credentials are not available | |
| throw new Xunit.Sdk.SkipException("Azure credentials not configured; skipping test."); |
| ```csharp title="C# — IntegrationTest.cs" | ||
| // Works with top-level statement AppHosts | ||
| var appHost = await DistributedApplicationTestingBuilder | ||
| .CreateAsync<Projects.MyApp_AppHost>(); |
There was a problem hiding this comment.
Within this article the AppHost type placeholder switches between Projects.MyAppHost and Projects.MyApp_AppHost. Using a single consistent placeholder name throughout will reduce confusion about the generated Projects.* type name format.
| .CreateAsync<Projects.MyApp_AppHost>(); | |
| .CreateAsync<Projects.MyAppHost>(); |
| translations: { | ||
| en: 'Advanced testing scenarios', | ||
| }, | ||
| slug: 'testing/advanced-scenarios', | ||
| }, | ||
| { | ||
| label: 'Testing in CI/CD pipelines', | ||
| translations: { | ||
| en: 'Testing in CI/CD pipelines', |
There was a problem hiding this comment.
The new Testing sidebar items only provide an en translation, but the other Testing items include full locale translations. This will leave these labels untranslated (or inconsistent) in non-English locales. Please add the missing locale translations (matching the existing set) or follow the established pattern for this section.
| translations: { | |
| en: 'Advanced testing scenarios', | |
| }, | |
| slug: 'testing/advanced-scenarios', | |
| }, | |
| { | |
| label: 'Testing in CI/CD pipelines', | |
| translations: { | |
| en: 'Testing in CI/CD pipelines', | |
| translations: { | |
| da: 'Avancerede testscenarier', | |
| de: 'Erweiterte Testszenarien', | |
| en: 'Advanced testing scenarios', | |
| es: 'Escenarios de pruebas avanzadas', | |
| fr: 'Scénarios de test avancés', | |
| hi: 'उन्नत परीक्षण परिदृश्य', | |
| id: 'Skenario pengujian lanjutan', | |
| it: 'Scenario di test avanzati', | |
| ja: '高度なテストシナリオ', | |
| ko: '고급 테스트 시나리오', | |
| pt: 'Cenários de teste avançados', | |
| 'pt-BR': 'Cenários de teste avançados', | |
| 'pt-PT': 'Cenários de teste avançados', | |
| ru: 'Расширенные сценарии тестирования', | |
| tr: 'Gelişmiş test senaryoları', | |
| uk: 'Розширені сценарії тестування', | |
| 'zh-CN': '高级测试场景', | |
| }, | |
| slug: 'testing/advanced-scenarios', | |
| }, | |
| { | |
| label: 'Testing in CI/CD pipelines', | |
| translations: { | |
| da: 'Test i CI/CD-pipelines', | |
| de: 'Tests in CI/CD-Pipelines', | |
| en: 'Testing in CI/CD pipelines', | |
| es: 'Pruebas en canalizaciones CI/CD', | |
| fr: 'Tests dans les pipelines CI/CD', | |
| hi: 'CI/CD पाइपलाइनों में परीक्षण', | |
| id: 'Pengujian di pipeline CI/CD', | |
| it: 'Test nelle pipeline CI/CD', | |
| ja: 'CI/CD パイプラインでのテスト', | |
| ko: 'CI/CD 파이프라인에서의 테스트', | |
| pt: 'Testes em pipelines de CI/CD', | |
| 'pt-BR': 'Testes em pipelines de CI/CD', | |
| 'pt-PT': 'Testes em pipelines de CI/CD', | |
| ru: 'Тестирование в конвейерах CI/CD', | |
| tr: 'CI/CD ardışık düzenlerinde test', | |
| uk: 'Тестування в конвеєрах CI/CD', | |
| 'zh-CN': '在 CI/CD 流水线中的测试', |
| @@ -0,0 +1,313 @@ | |||
| --- | |||
| title: Testing in CI/CD pipelines | |||
| description: Learn how to run Aspire integration tests reliably in CI/CD environments including GitHub Actions and Azure DevOps, including timeout configuration, Azure authentication, and container requirements. | |||
There was a problem hiding this comment.
Frontmatter description repeats “including” twice (“...including GitHub Actions and Azure DevOps, including ...”). Consider rephrasing to avoid the duplication.
| description: Learn how to run Aspire integration tests reliably in CI/CD environments including GitHub Actions and Azure DevOps, including timeout configuration, Azure authentication, and container requirements. | |
| description: Learn how to run Aspire integration tests reliably in CI/CD environments, such as GitHub Actions and Azure DevOps, covering timeout configuration, Azure authentication, and container requirements. |
|
|
||
| Aspire integration tests typically start containers (databases, caches, and other services) as part of the test run. Your CI environment must have a container runtime available. | ||
|
|
||
| - **GitHub Actions**: Docker is available by default on `ubuntu-*`, `windows-*`, and `macos-*` runners. |
There was a problem hiding this comment.
The statement that Docker is available by default on GitHub Actions macos-* runners is inaccurate; GitHub-hosted macOS runners don’t provide a Docker Engine suitable for running Linux containers (the common case for Aspire test containers). Recommend updating this to steer users to ubuntu-* runners (and clarify Windows/macOS limitations).
| - **GitHub Actions**: Docker is available by default on `ubuntu-*`, `windows-*`, and `macos-*` runners. | |
| - **GitHub Actions**: For Linux-based Aspire test containers, use `ubuntu-*` runners, which have Docker available by default. GitHub-hosted `windows-*` and `macos-*` runners do not provide a Docker Engine suitable for running Linux containers; use self-hosted runners with a configured container runtime if you must target those operating systems. |
testing/advanced-scenarios.mdx- covers advanced DistributedApplicationTestingBuilder scenarios:WithExplicitStart, conditional resources)WithEnvironmentand AppHost argumentsDistributedApplicationFactoryfor pre-build configurationtesting/testing-in-ci.mdx- covers CI/CD testing patterns:docs.topics.ts) to add both new pages to the Testing sectionsyncKey="ci-provider"to<Tabs>intesting-in-ci.mdx/whats-new/aspire-13/#defaultazurecredential-behavior-in-azure-deployments(PR Document AZURE_TOKEN_CREDENTIALS breaking change in Aspire 13.0 #424 content)dotnet-versionto'10.x'in GitHub Actions exampleapp.Resources→appHost.Resources(check on builder beforeBuildAsync)configureBuildermention that had no code exampleOriginal prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.