Skip to content

Add setup-local-sdk skill for global.json paths feature#508

Open
jfversluis wants to merge 7 commits into
mainfrom
skills/setup-local-sdk
Open

Add setup-local-sdk skill for global.json paths feature#508
jfversluis wants to merge 7 commits into
mainfrom
skills/setup-local-sdk

Conversation

@jfversluis
Copy link
Copy Markdown
Member

Summary

Adds a new skill that guides users through installing a .NET SDK into a project-local directory using the global.json paths feature (new in .NET 10).

Reopened from #506 (fork PR) to allow eval judges to access CI secrets.

Files

File Description
plugins/dotnet/skills/setup-local-sdk/SKILL.md 250-line skill with 12-step workflow
tests/dotnet/setup-local-sdk/eval.yaml 5 eval scenarios
.github/CODEOWNERS Added entry for @jfversluis and @redth

What the skill covers

  • Verifying .NET 10+ host prerequisite
  • Installing a prerelease/specific SDK with dotnet-install scripts
  • Configuring global.json with paths and $host$
  • Installing workloads (MAUI, wasm-tools) on the local SDK
  • Cross-platform team install scripts
  • Updating .gitignore
  • Verification and cleanup guidance

Key design decisions

  • Workload commands always use ./.dotnet/dotnet rather than the system dotnet. Testing revealed that global.json paths routes SDK resolution correctly for build/run/test, but workload metadata is stored relative to the host's dotnet root, not the resolved SDK root (dotnet/sdk#49825).
  • No Aspire workload references since Aspire 9+ is NuGet package-based and no longer requires a workload.
  • .NET 10+ is a hard requirement -- no fallback guidance for older hosts.
  • All instructions tested against real .NET 11 preview.2 on macOS.

Related

jfversluis and others added 6 commits April 8, 2026 15:05
Adds a skill that guides users through installing a .NET SDK into a
project-local directory using the global.json paths feature (.NET 10+).

Includes:
- 12-step workflow: verify host, install SDK, configure global.json,
  gitignore, workloads, team scripts, verification
- MAUI and wasm-tools workload support
- Cross-platform install scripts (bash + PowerShell)
- 7 eval scenarios covering basic setup, exact version, team scripts,
  troubleshooting, incompatible host, existing .dotnet/, and MAUI workload
- CODEOWNERS entry for @jfversluis and @Redth

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add Windows PowerShell equivalents for version check (Step 6) and
  workload list commands
- Fix rollForward description: latestFeature rolls across feature bands,
  not just patches
- Add global.json backup in team install scripts before overwriting
- Fix eval scenario: provide explicit host version (9.0.306) in prompt
  so the incompatible-host test is deterministic

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SKILL.md:
- Windows -Version flag for exact installs (vs --version on bash)
- Workload note includes both ./.dotnet/dotnet and .\.dotnet\dotnet.exe
- Cleanup includes Windows Remove-Item equivalent
- Revert/delete instructions include both OS forms

eval.yaml:
- Remove brittle --version assertion; use rubric for version flag check
- Increase all scenario timeouts from 120s to 180s (many were timing out)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 239 lines, -52%):
- Rewrite description: intent-focused USE FOR/DO NOT USE FOR pattern
  with activation keywords (MAUI, existing, testing, team)
- Remove personas table, checkpoint markers, verbose notes
- Condense all sections while preserving complete workflow
 5 entries)
- Remove redundant Validation section

 5 scenarios):
- Drop 'Handle existing' (handled naturally by Step 4)
- Drop 'Verify SDK resolution' (covered by basic setup rubric)
- Add expect_tools: ['bash'] to actionable scenarios
- Reduce rubric items to 3-4 per scenario
- Incompatible host scenario: 60s timeout (quick response)

Expected improvements:
- Token usage: -50% (skill is half the size)
- Activation: USE FOR keywords match all prompts
- Variance: fewer scenarios = less variability
- Eval time: -40% (fewer scenarios, shorter timeouts)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- MINGW/MSYS/CYGWIN treated as bash-capable (Git Bash), not PowerShell
- Remove hardcoded 'Install directory' input row
- Make allowPrerelease conditional on preview installs
- Make errorMessage conditional on team scripts being created
- Add PowerShell equivalent for .gitignore update
- Add assertion to incompatible host eval scenario

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 8, 2026 14:21
@jfversluis
Copy link
Copy Markdown
Member Author

/evaluate

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new dotnet skill documenting how to install and use a project-local .NET SDK via global.json paths (requires .NET 10+ host), along with evaluation scenarios and CODEOWNERS coverage for the new folders.

Changes:

  • Added setup-local-sdk skill documentation describing a 12-step workflow (install, global.json configuration, workloads, team scripts, cleanup).
  • Added eval scenarios for the new skill under tests/dotnet/setup-local-sdk/.
  • Added CODEOWNERS entries for the new skill and test directories.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
plugins/dotnet/skills/setup-local-sdk/SKILL.md New skill doc guiding local SDK install + global.json paths configuration + workload/team-script guidance.
tests/dotnet/setup-local-sdk/eval.yaml New eval scenarios validating expected guidance for local SDK setup.
.github/CODEOWNERS Adds owners for the new skill/test directories.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread plugins/dotnet/skills/setup-local-sdk/SKILL.md Outdated
Comment thread plugins/dotnet/skills/setup-local-sdk/SKILL.md Outdated
Comment thread plugins/dotnet/skills/setup-local-sdk/SKILL.md
Comment thread tests/dotnet/setup-local-sdk/eval.yaml Outdated
Comment thread tests/dotnet/setup-local-sdk/eval.yaml Outdated
github-actions Bot added a commit that referenced this pull request Apr 8, 2026
@github-actions

This comment was marked as outdated.

Immediate fixes based on eval results analysis:

eval.yaml:
 120s)
  Rationale: All 5 scenarios timed out; runs need more time to produce output
- Remove expect_tools constraints: redundant with timeout fix and brittle
- Change incompatible-host assertion from '.10.' to '.NET 10.' for specificity

SKILL.md:
 -fsSL (fail fast on HTTP errors)
- Improve global.json merge guidance: explicitly document backing up and preserving
  existing msbuild-sdks/tools properties
- Rationale: Addresses 5 unresolved PR review comments

Key findings from eval artifact analysis:
- Plugin mode produces correct output (skill is good)
- Isolated mode times out + no output (timeout is blocker)
- Judge JSON-RPC failures are infrastructure issue (not our problem)
- Skill activates correctly in all scenarios

Next: Wait for CI to run with longer timeouts, then address judge infrastructure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jfversluis
Copy link
Copy Markdown
Member Author

/evaluate

@jfversluis
Copy link
Copy Markdown
Member Author

Analysis of Eval Results

I've analyzed the eval artifacts from run 24140442696 using the investigation guide. The 3.0/5 scores mask real skill quality.

Root Causes

1. Timeout blocks everything (FIXED)

  • All 5 scenarios hit the old 60-180s timeout limit
  • Agents were terminated before producing output
  • Fix: Updated eval.yaml with 300s timeouts for standard scenarios, 120s for incompatible-host
  • Status: New commit with longer timeouts pushed; awaiting re-evaluation

2. Judge infrastructure failures (NOT OUR PROBLEM)

  • Judge JSON-RPC connection failures on 100% of baseline + isolated + plugin runs
  • This causes all scores to default to 3.0/5 (infrastructure fallback)
  • Same issue affects ALL evals in the dotnet/skills repo, including main branch
  • Impact: True quality scores are masked until judges work
  • Status: Blocking all meaningful scoring across the repo

3. Plugin mode works perfectly (STRONG SIGNAL)
.dotnet/, global.json with paths, .gitignore updated, cleanup instructions)

  • This is exactly what we want
  • Implication: Skill content is sound; eval environment is the bottleneck

Changes Made

300s, removed brittle expect_tools constraints, improved incompatible-host assertion specificity

-fsSL for fail-fast), improved global.json merge guidance per review comments

Next Steps

Await re-evaluation with 300s timeouts1.
Judge failures are infrastructure-level; waiting for dotnet/skills team to resolve3. 2.
4 Plugin mode results should show real quality once timeouts clear and judges work.

Summary: The skill works (plugin mode proves it). Scores are suppressed by timeout + judge infrastructure issues. We've fixed what we control (timeouts). The rest requires infrastructure support.

@jfversluis
Copy link
Copy Markdown
Member Author

Status Update (15:02 UTC)

All 5 review comments resolved and addressed:
-fsSL (fail fast on HTTP errors)

  • global.json merge guidance: improved documentation

  • eval.yaml timeouts: increased to 300s (was 180s), removed brittle expect_tools constraints

  • Previous eval (14:21, old 180s timeouts): All 3.0/5 due to timeouts

  • New eval (14:58, 300s timeouts): Awaiting results

  • Judge infrastructure: JSON-RPC failures on 100% of runs (not our problem, systemic issue)

Awaiting re-evaluation with longer timeouts to see if isolated mode can now produce output. Will address any new issues that arise.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 8, 2026

Skill Validation Results

Skill Scenario Quality Skills Loaded Overfit Verdict
setup-local-sdk Basic local SDK setup with .NET 11 preview 3.0/5 ⏰ → 3.0/5 ⏰ ✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: skill, create [1]
setup-local-sdk Install a specific SDK version locally 3.0/5 ⏰ → 3.0/5 ⏰ ✅ setup-local-sdk; tools: skill [2]
setup-local-sdk Set up local SDK with MAUI workload 3.0/5 ⏰ → 3.0/5 ⏰ ✅ setup-local-sdk; tools: skill, read_bash / ✅ setup-local-sdk; tools: skill, create, read_bash [3]
setup-local-sdk Create team install scripts 3.0/5 ⏰ → 3.0/5 ⏰ ✅ setup-local-sdk; tools: skill / ✅ setup-local-sdk; tools: skill, create [4]
setup-local-sdk Detect incompatible .NET host version 3.0/5 ⏰ → 3.0/5 ✅ setup-local-sdk; tools: skill

[1] ⚠️ High run-to-run variance (CV=1.43) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -2.1% due to: tool calls (7 → 10), tokens (74086 → 89016)
[2] (Isolated) Quality unchanged but weighted score is -1.3% due to: tool calls (6 → 8)
[3] ⚠️ High run-to-run variance (CV=4.16) — consider re-running with --runs 5
[4] ⚠️ High run-to-run variance (CV=1.29) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -2.3% due to: tokens (68425 → 94923)

timeout — run(s) hit the (120s, 300s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

@github-actions
Copy link
Copy Markdown
Contributor

This PR has been automatically marked as stale because it has no activity for 30 days. It will be closed if no further activity occurs within another 7 days of this comment. If it is closed, you may reopen it anytime when you're ready again.

Note

🔒 Integrity filter blocked 29 items

The following items were blocked because they don't meet the GitHub integrity level.

  • #623 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #617 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #609 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #607 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #601 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #598 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #502 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #486 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #376 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #358 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #357 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #329 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #303 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #269 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #267 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #266 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • ... and 13 more items

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by Close Stale Pull Requests · ● 1.1M ·

@github-actions
Copy link
Copy Markdown
Contributor

This pull request has been automatically closed because it has been open for more than 30 days with no recent activity.

If you believe this work is still relevant, please feel free to reopen or create a new pull request. Thank you for your contribution!

Note

🔒 Integrity filter blocked 30 items

The following items were blocked because they don't meet the GitHub integrity level.

  • #653 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #644 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #623 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #609 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #607 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #601 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #598 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #502 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #486 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #376 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #358 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #357 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #329 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #303 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #269 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #267 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • ... and 14 more items

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by Close Stale Pull Requests · ● 439.4K

@github-actions github-actions Bot closed this May 18, 2026
@Redth Redth reopened this May 21, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Skill Coverage Report

Plugin Skill Covered Coverage
⚠️ dotnet-ai mcp-csharp-create 18/25 72%
dotnet-ai mcp-csharp-debug 16/18 88.9%
⚠️ dotnet-ai mcp-csharp-publish 9/15 60%
⚠️ dotnet-ai mcp-csharp-test 11/18 61.1%
⚠️ dotnet-ai technology-selection 18/24 75%
⚠️ dotnet-aspnet configuring-opentelemetry-dotnet 13/21 61.9%
dotnet-aspnet dotnet-webapi error
⚠️ dotnet-aspnet minimal-api-file-upload 4/8 50%
dotnet-data optimizing-ef-core-queries 4/17 23.5%
⚠️ dotnet-diag analyzing-dotnet-performance 12/21 57.1%
dotnet-diag android-tombstone-symbolication 5/5 100%
dotnet-diag apple-crash-symbolication error
dotnet-diag clr-activation-debugging 16/19 84.2%
dotnet-diag dotnet-trace-collect 15/17 88.2%
dotnet-diag microbenchmarking 3/3 100%
dotnet-experimental exp-assertion-quality 16/18 88.9%
dotnet-experimental exp-dotnet-test-frameworks 0/5 0%
dotnet-experimental exp-mock-usage-analysis 12/13 92.3%
⚠️ dotnet-experimental exp-test-gap-analysis 14/21 66.7%
⚠️ dotnet-experimental exp-test-maintainability 10/18 55.6%
⚠️ dotnet-experimental exp-test-smell-detection 15/19 78.9%
⚠️ dotnet-experimental exp-test-tagging 14/21 66.7%
⚠️ dotnet-maui maui-collectionview 7/9 77.8%
⚠️ dotnet-maui maui-data-binding 6/10 60%
dotnet-maui maui-dependency-injection 0/1 0%
dotnet-maui maui-safe-area 0/2 0%
dotnet-maui maui-theming 0/1 0%
dotnet-nuget convert-to-cpm 16/16 100%
dotnet-template-engine template-authoring 5/16 31.2%
dotnet-template-engine template-discovery 10/11 90.9%
⚠️ dotnet-template-engine template-instantiation 11/16 68.8%
⚠️ dotnet-template-engine template-validation 6/10 60%
⚠️ dotnet-upgrade dotnet-aot-compat 6/10 60%
dotnet-upgrade migrate-dotnet10-to-dotnet11 5/6 83.3%
dotnet-upgrade migrate-dotnet8-to-dotnet9 6/6 100%
dotnet-upgrade migrate-dotnet9-to-dotnet10 6/6 100%
⚠️ dotnet-upgrade migrate-nullable-references 20/27 74.1%
dotnet-upgrade thread-abort-migration 21/22 95.5%
dotnet csharp-scripts 1/16 6.2%
dotnet dotnet-pinvoke 6/28 21.4%
⚠️ dotnet setup-local-sdk 3/5 60%
dotnet-msbuild binlog-failure-analysis 4/5 80%
dotnet-msbuild directory-build-organization 0/1 0%
⚠️ dotnet-msbuild msbuild-modernization 5/7 71.4%
⚠️ dotnet-msbuild msbuild-server 7/9 77.8%
dotnet-msbuild resolve-project-references 5/6 83.3%
dotnet-test code-testing-agent 4/5 80%
dotnet-test crap-score 6/6 100%
dotnet-test dotnet-test-frameworks 0/5 0%
dotnet-test filter-syntax 0/1 0%
⚠️ dotnet-test migrate-mstest-v1v2-to-v3 12/16 75%
dotnet-test migrate-mstest-v3-to-v4 34/41 82.9%
dotnet-test migrate-vstest-to-mtp 20/24 83.3%
dotnet-test migrate-xunit-to-xunit-v3 18/19 94.7%
dotnet-test mtp-hot-reload 16/16 100%
dotnet-test run-tests 14/15 93.3%
dotnet-test test-anti-patterns 16/18 88.9%
⚠️ dotnet-test writing-mstest-tests 42/62 67.7%
dotnet11 `` error
Uncovered: dotnet-ai/mcp-csharp-create
  • [Validation] Project builds with no errors (dotnet build) (line 231)
  • [Validation] HTTP: app.MapMcp() is called in Program.cs (line 236)
  • [Validation] Server starts successfully with dotnet run (line 237)
  • [Pitfall] WithToolsFromAssembly() fails in AOT (line 246)
  • [WorkflowStep] Step 5: Add prompts and resources (optional) (line 150)
  • [CodePattern] CancellationToken (line 89)
  • [CodePattern] [Description] (line 89)
Uncovered: dotnet-ai/mcp-csharp-debug
  • [Validation] Breakpoints hit when debugging in IDE (line 184)
  • [Pitfall] HTTP server returns 404 at MCP endpoint (line 194)
Uncovered: dotnet-ai/mcp-csharp-publish
  • [Validation] NuGet: Package installs and runs via dnx PackageId@version (line 242)
  • [Validation] Azure: Server is reachable and tools respond (line 244)
  • [Validation] MCP Registry: Server appears at registry.modelcontextprotocol.io (line 245)
  • [Validation] MCP client can connect and call tools on the deployed server (line 246)
  • [Pitfall] Docker container exits immediately (line 254)
  • [Pitfall] API keys leaked in Docker image (line 257)
Uncovered: dotnet-ai/mcp-csharp-test
  • [Validation] All tests pass: dotnet test (line 164)
  • [Validation] Tests run in CI without manual setup (line 165)
  • [Pitfall] Full test suite runs are slow (line 175)
  • [WorkflowStep] Step 4: Run tests (line 137)
  • [CodePattern] [Fact] (line 56)
  • [CodePattern] [Theory] (line 56)
  • [CodePattern] [InlineData] (line 56)
Uncovered: dotnet-ai/technology-selection
  • [Validation] API keys are loaded from secure sources — not in source code or committed config files (line 313)
  • [Validation] Non-deterministic outputs have validation and fallback paths (line 318)
  • [Validation] dotnet build -c Release -warnaserror completes cleanly (line 319)
  • [Pitfall] Over-engineering with LLMs (line 347)
  • [Pitfall] Cold start latency on ML.NET models (line 353)
  • [WorkflowStep] Step 3: Implement with guardrails (line 133)
Uncovered: dotnet-aspnet/configuring-opentelemetry-dotnet
  • [Validation] Traces appear in the observability backend (Jaeger, Aspire dashboard, etc.) (line 271)
  • [Validation] HTTP requests automatically create spans with correct verb, URL, status code (line 272)
  • [Validation] Health check endpoints are filtered from traces (line 276)
  • [Validation] Exception details appear on error spans (line 277)
  • [Pitfall] Missing HTTP client spans (line 285)
  • [Pitfall] High cardinality tags (line 286)
  • [CodePattern] [key] (line 240)
  • [CodePattern] readonly (line 123)
Uncovered: dotnet-aspnet/minimal-api-file-upload
  • [WorkflowStep] Step 1: CRITICAL — Understand IFormFile Binding in Minimal APIs (line 28)
  • [WorkflowStep] Step 5: CRITICAL — Streaming Large Files Without Buffering (line 169)
  • [CodePattern] [FromForm] (line 30)
  • [CodePattern] [RequestSizeLimit] (line 53)
Uncovered: dotnet-data/optimizing-ef-core-queries
  • [Validation] SQL logging shows expected number of queries (no N+1) (line 163)
  • [Validation] Read-only queries use AsNoTracking() (line 164)
  • [Validation] Hot-path queries use compiled queries (line 165)
  • [Validation] No client-side evaluation warnings in logs (line 166)
  • [Validation] Include/split strategy matches data shape (line 167)
  • [Pitfall] Global query filters forgotten in perf analysis (line 174)
  • [Pitfall] DbContext kept alive too long (line 175)
  • [Pitfall] String interpolation in FromSqlRaw (line 177)
  • [WorkflowStep] Step 1: Enable query logging to see the actual SQL (line 30)
  • [WorkflowStep] Step 4: Use compiled queries for hot paths (line 119)
  • [WorkflowStep] Step 5: Avoid common query traps (line 133)
  • [WorkflowStep] Step 6: Use raw SQL or FromSql for complex queries (line 143)
  • [CodePattern] readonly (line 121)
Uncovered: dotnet-diag/analyzing-dotnet-performance
  • [Validation] All critical patterns were checked (from reference files or inline recipes) (line 177)
  • [Validation] Topic-specific recipes run only when matching signals detected (line 178)
  • [Validation] Each finding includes a concrete code fix (line 179)
  • [Validation] Scan execution checklist is complete (all recipes run) (line 180)
  • [Validation] Summary table included at end (line 181)
  • [Pitfall] Suggesting ConfigureAwait(false) in app code (line 190)
  • [Pitfall] Recommending ValueTask everywhere (line 191)
  • [Pitfall] Flagging new HttpClient() in DI services (line 192)
  • [Pitfall] Suggesting CollectionsMarshal.AsSpan broadly (line 194)
Uncovered: dotnet-diag/clr-activation-debugging
  • [Validation] The entry point for each problematic activation was identified (line 288)
  • [Validation] SEM_FAILCRITICALERRORS state was noted for FOD-related issues (line 291)
  • [Validation] Multiple activations within a single log were individually traced (line 292)
Uncovered: dotnet-diag/dotnet-trace-collect
  • [Validation] The recommended tool is compatible with the developer's runtime, OS, and deployment topology (line 234)
  • [Validation] The output file is generated in the expected location (line 236)
Uncovered: dotnet-experimental/exp-assertion-quality
  • [Validation] Metrics are computed correctly (counts add up) (line 140)
  • [Validation] If the suite has good diversity, the report acknowledges this (line 145)
Uncovered: dotnet-experimental/exp-dotnet-test-frameworks
  • [CodePattern] Assert.Equal (line 74)
  • [CodePattern] Assert.That (line 82)
  • [CodePattern] Assert.AreEqual (line 60)
  • [CodePattern] Assert.ThrowsExactly (line 60)
  • [CodePattern] Assert.Throws (line 60)
Uncovered: dotnet-experimental/exp-mock-usage-analysis
  • [Validation] Production code was read and execution paths were traced (not just test code reviewed) (line 85)
Uncovered: dotnet-experimental/exp-test-gap-analysis
  • [Validation] Trivial code (simple getters, auto-properties) is excluded from analysis (line 191)
  • [Validation] Findings are prioritized by risk, not just listed in source order (line 192)
  • [Validation] Report includes strengths (killed mutations) alongside gaps (line 193)
  • [Validation] Mutation categories are correctly labeled (line 194)
  • [Pitfall] Analyzing trivial code (line 200)
  • [Pitfall] Ignoring call chains (line 202)
  • [Pitfall] Over-counting mutations in generated code (line 203)
Uncovered: dotnet-experimental/exp-test-maintainability
  • [Validation] Every finding shows the actual duplicated code, not just a description (line 180)
  • [Validation] Every suggestion includes a concrete before/after example (line 181)
  • [Validation] Findings are filtered through the 3+ occurrence threshold (line 182)
  • [Validation] Simple constructors are not flagged (line 183)
  • [Validation] If tests are clean, the report says so upfront (line 185)
  • [Pitfall] Suggesting extraction for 2 occurrences (line 192)
  • [Pitfall] Flagging simple new X() as boilerplate (line 195)
  • [CodePattern] [TestMethod] (line 59)
Uncovered: dotnet-experimental/exp-test-smell-detection
  • [Validation] Every finding includes a concrete fix example (not just "fix this") (line 180)
  • [Validation] Contextually obvious numbers are not flagged as magic numbers (line 183)
  • [Validation] Severity levels are justified, not arbitrary (line 185)
  • [Pitfall] Treating skip annotations with reasons same as bare skips (line 196)
Uncovered: dotnet-experimental/exp-test-tagging
  • [Validation] No invented trait values outside the taxonomy table (line 161)
  • [Validation] Existing trait attributes were preserved, not duplicated (line 162)
  • [Validation] The project still builds after changes (dotnet build) (line 164)
  • [Pitfall] Guessing traits without reading the test body (line 170)
  • [WorkflowStep] Step 1: Detect the test framework (line 57)
  • [WorkflowStep] Step 3: Classify each test method (line 73)
  • [CodePattern] [Category] (line 116)
Uncovered: dotnet-maui/maui-collectionview
  • [Pitfall] EmptyView doesn't render correctly (line 328)
  • [Pitfall] Poor scroll performance (line 329)
Uncovered: dotnet-maui/maui-data-binding
  • [Pitfall] Specifying redundant Mode=OneWay / Mode=TwoWay (line 374)
  • [Pitfall] Mutating ObservableCollection off the UI thread (line 376)
  • [Pitfall] Complex converter chains in hot paths (line 377)
  • [Pitfall] Binding to non-public properties (line 379)
Uncovered: dotnet-maui/maui-dependency-injection
  • [CodePattern] readonly (line 101)
Uncovered: dotnet-maui/maui-safe-area
  • [CodePattern] [Flags] (line 47)
  • [CodePattern] readonly (line 64)
Uncovered: dotnet-maui/maui-theming
  • [CodePattern] [Activity] (line 225)
Uncovered: dotnet-template-engine/template-authoring
  • [Validation] Template identity and shortName are unique and meaningful (line 98)
  • [Validation] All parameters have descriptions and appropriate defaults (line 99)
  • [Validation] Template can be installed, dry-run, and instantiated successfully (line 100)
  • [Validation] Created projects build cleanly with dotnet build (line 101)
  • [Validation] Conditional content produces correct output for all parameter combinations (line 102)
  • [Pitfall] Identity format issues (line 108)
  • [Pitfall] Missing parameter descriptions (line 110)
  • [Pitfall] Not testing all parameter combinations (line 111)
  • [Pitfall] Not setting classifications (line 113)
  • [WorkflowStep] Step 3: Refine the template (line 76)
  • [WorkflowStep] Step 4: Test the template locally (line 86)
Uncovered: dotnet-template-engine/template-discovery
  • [Validation] User understands what the template produces before proceeding to creation (line 102)
Uncovered: dotnet-template-engine/template-instantiation
  • [Validation] If CPM is active, .csproj has no version attributes and Directory.Packages.props has matching entries (line 121)
  • [Pitfall] Creating projects without specifying the framework (line 130)
  • [WorkflowStep] Step 2: Analyze the workspace (line 49)
  • [WorkflowStep] Step 3: Preview the creation (line 58)
  • [WorkflowStep] Step 6: Template package management (line 102)
Uncovered: dotnet-template-engine/template-validation
  • [Pitfall] Invalid datatype value (line 152)
  • [Pitfall] Parameter prefix collision (Auth vs AuthMode) (line 154)
  • [Pitfall] Source condition without parentheses (line 155)
  • [WorkflowStep] Step 1: Locate the template.json (line 123)
Uncovered: dotnet-upgrade/dotnet-aot-compat
  • [CodePattern] [RequiresUnreferencedCode] (line 194)
  • [CodePattern] [DynamicallyAccessedMembers] (line 118)
  • [CodePattern] [JsonSerializerContext] (line 168)
  • [CodePattern] [MSBuild] (line 66)
Uncovered: dotnet-upgrade/migrate-dotnet10-to-dotnet11
  • [WorkflowStep] Step 5: Update infrastructure (line 139)
Uncovered: dotnet-upgrade/migrate-nullable-references
  • [Validation] Project file(s) contain <Nullable>enable</Nullable> (or #nullable enable per-file for file-by-file strategy) (line 234)
  • [Validation] <WarningsAsErrors>nullable</WarningsAsErrors> added to project file to prevent regressions (line 236)
  • [Validation] No #nullable disable directives remain unless justified with a comment (line 238)
  • [Validation] Public API signatures accurately reflect null contracts (line 240)
  • [Validation] For public libraries: breaking changes documented in nullable-breaking-changes.md and reviewed by the user (line 241)
  • [Pitfall] Multi-target projects and older TFMs (line 264)
  • [Pitfall] Warnings reappear after upgrading a dependency (line 265)
Uncovered: dotnet-upgrade/thread-abort-migration
  • [Validation] No SYSLIB0006 pragma suppressions remain (line 116)
Uncovered: dotnet/csharp-scripts
  • [Validation] dotnet --version reports 10.0 or later (or fallback path is used) (line 194)
  • [Validation] The script compiles without errors (can be checked explicitly with dotnet build <file>.cs) (line 195)
  • [Validation] dotnet <file>.cs produces the expected output (line 196)
  • [Validation] Script file and cached artifacts are cleaned up after the session (line 197)
  • [Pitfall] .cs file is inside a directory with a .csproj (line 203)
  • [Pitfall] #:package without a version (line 204)
  • [Pitfall] #:property with wrong syntax (line 205)
  • [Pitfall] Directives placed after C# code (line 206)
  • [Pitfall] Reflection-based JSON serialization fails (line 207)
  • [Pitfall] Unexpected build behavior or version errors (line 208)
  • [WorkflowStep] Step 1: Check the .NET SDK version (line 27)
  • [WorkflowStep] Step 2: Write the script file (line 31)
  • [WorkflowStep] Step 4: Add directives (if needed) (line 61)
  • [WorkflowStep] Step 5: Clean up (line 119)
  • [CodePattern] [MSBuild] (line 89)
Uncovered: dotnet/dotnet-pinvoke
  • [Validation] Calling convention specified if targeting Windows x86; omitted otherwise (line 415)
  • [Validation] String encoding is explicit — no reliance on defaults or CharSet.Auto (line 416)
  • [Validation] Memory ownership is documented and matched (who allocates, who frees, with what) (line 417)
  • [Validation] SafeHandle used for all native handles (no raw IntPtr escaping the interop layer) (line 418)
  • [Validation] Delegates passed as callbacks are rooted to prevent GC collection (line 419)
  • [Validation] SetLastError/SetLastPInvokeError set for APIs that use OS error codes (line 420)
  • [Validation] Struct layout matches native (packing, alignment, field order) (line 421)
  • [Validation] CLong/CULong used for C long/unsigned long in cross-platform code (line 422)
  • [Validation] If using CLong/CULong with LibraryImport, [assembly: DisableRuntimeMarshalling] is applied (line 423)
  • [Validation] No bool without explicit MarshalAs — always specify UnmanagedType.Bool (4-byte) or UnmanagedType.U1 (1-byte) to ensure normalization across the language boundary. (line 424)
  • [WorkflowStep] Step 5: Establish Memory Ownership (line 167)
  • [WorkflowStep] Step 6: Use SafeHandle for Native Handles (line 223)
  • [WorkflowStep] Step 7: Handle Errors (line 260)
  • [CodePattern] [UnmanagedCallConv] (line 110)
  • [CodePattern] [DllImport] (line 92)
  • [CodePattern] [Out] (line 143)
  • [CodePattern] [LibraryImport] (line 100)
  • [CodePattern] [UnmanagedCallersOnly] (line 280)
  • [CodePattern] sealed (line 227)
  • [CodePattern] [UnmanagedFunctionPointer] (line 299)
  • [CodePattern] [256] (line 175)
  • [CodePattern] [MarshalAs] (line 143)
Uncovered: dotnet/setup-local-sdk
  • [Pitfall] paths ignored (line 248)
  • [Pitfall] dotnet app.dll wrong runtime (line 252)
Uncovered: dotnet-msbuild/binlog-failure-analysis
  • [WorkflowStep] Step 3: Search for context around specific errors (line 33)
Uncovered: dotnet-msbuild/directory-build-organization
  • [CodePattern] [MSBuild] (line 122)
Uncovered: dotnet-msbuild/msbuild-modernization
  • [WorkflowStep] Step 6: Remove Unnecessary Boilerplate (line 216)
  • [WorkflowStep] Step 7: Enable Modern Features (line 275)
Uncovered: dotnet-msbuild/msbuild-server
  • [Validation] MSBUILDUSESERVER=1 is set in the shell (line 58)
  • [Validation] Second sequential build is faster than the first (line 59)
Uncovered: dotnet-msbuild/resolve-project-references
  • [Validation] ResolveProjectReferences was not set as the optimization target (line 57)
Uncovered: dotnet-test/code-testing-agent
  • [WorkflowStep] Step 2: Invoke the Test Generator (line 72)
Uncovered: dotnet-test/dotnet-test-frameworks
  • [CodePattern] Assert.Equal (line 74)
  • [CodePattern] Assert.That (line 82)
  • [CodePattern] Assert.AreEqual (line 60)
  • [CodePattern] Assert.ThrowsExactly (line 60)
  • [CodePattern] Assert.Throws (line 60)
Uncovered: dotnet-test/filter-syntax
  • [CodePattern] [trait] (line 112)
Uncovered: dotnet-test/migrate-mstest-v1v2-to-v3
  • [Validation] Project builds with zero errors (line 184)
  • [Validation] All tests pass (dotnet test) -- compare pass/fail counts to pre-migration baseline (line 185)
  • [CodePattern] Assert.AreNotEqual (line 143)
  • [CodePattern] Assert.AreSame (line 143)
Uncovered: dotnet-test/migrate-mstest-v3-to-v4
  • [Validation] Project builds with zero errors (line 447)
  • [Validation] All tests pass with dotnet test (line 448)
  • [Validation] Behavioral changes reviewed and addressed (line 455)
  • [CodePattern] [TestMethodAttribute] (line 173)
  • [CodePattern] [Timeout] (line 213)
  • [CodePattern] sealed (line 119)
  • [CodePattern] [TestMethod] (line 266)
Uncovered: dotnet-test/migrate-vstest-to-mtp
  • [Validation] dotnet build completes with zero errors (line 311)
  • [Validation] Test executable runs directly (e.g., ./bin/Debug/net8.0/MyTests.exe) (line 313)
  • [Validation] No vstest.console.exe invocations remain in CI scripts (line 316)
  • [CodePattern] [trait] (line 202)
Uncovered: dotnet-test/migrate-xunit-to-xunit-v3
  • [CodePattern] sealed (line 114)
Uncovered: dotnet-test/run-tests
  • [Validation] Correct dotnet test invocation was used for the detected platform and SDK version (line 190)
Uncovered: dotnet-test/test-anti-patterns
  • [Validation] Every finding includes a specific location (not just a general warning) (line 120)
  • [Validation] Recommendations are prioritized by severity (line 124)
Uncovered: dotnet-test/writing-mstest-tests
  • [Validation] Project builds with zero errors and all tests pass (line 332)
  • [WorkflowStep] Step 7: Use advanced features where appropriate (line 289)
  • [CodePattern] Assert.EndsWith (line 157)
  • [CodePattern] Assert.IsLessThan (line 177)
  • [CodePattern] Assert.DoesNotContain (line 136)
  • [CodePattern] Assert.MatchesRegex (line 157)
  • [CodePattern] Assert.IsInRange (line 177)
  • [CodePattern] Assert.AreSame (line 112)
  • [CodePattern] Parallelize (line 315)
  • [CodePattern] Assert.IsNull (line 112)
  • [CodePattern] [TestCleanup] (line 238)
  • [CodePattern] Assert.IsEmpty (line 136)
  • [CodePattern] [CICondition] (line 303)
  • [CodePattern] MSTest.Sdk (line 45)
  • [CodePattern] Assert.IsGreaterThan (line 177)
  • [CodePattern] [DoNotParallelize] (line 315)
  • [CodePattern] [OSCondition] (line 303)
  • [CodePattern] DoNotParallelize (line 315)
  • [CodePattern] [Retry] (line 295)
  • [CodePattern] Assert.StartsWith (line 157)

@AbhitejJohn
Copy link
Copy Markdown
Contributor

Tagging @dotnet/skills-csharp-language-reviewers to help review.

@AbhitejJohn AbhitejJohn added the waiting-on-review PR state label label May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants