Skip to content

Add binlog MCP usage to other msbuild skills/agents#683

Merged
JanKrivanek merged 2 commits into
mainfrom
dev/jankrivanek/msbuild-skills-use-mcp
May 22, 2026
Merged

Add binlog MCP usage to other msbuild skills/agents#683
JanKrivanek merged 2 commits into
mainfrom
dev/jankrivanek/msbuild-skills-use-mcp

Conversation

@JanKrivanek
Copy link
Copy Markdown
Member

Followup of #673

Using binlog mcp in other msbuild skills/agents

@JanKrivanek JanKrivanek requested a review from ViktorHofer as a code owner May 22, 2026 09:34
Copilot AI review requested due to automatic review settings May 22, 2026 09:34
@JanKrivanek JanKrivanek requested a review from a team as a code owner May 22, 2026 09:34
@JanKrivanek
Copy link
Copy Markdown
Member Author

/evaluate

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the dotnet-msbuild plugin’s existing binlog MCP integration by updating additional MSBuild skills and the build performance agent to prefer structured MCP binlog analysis (with a text-log replay fallback).

Changes:

  • Added “Primary: binlog MCP (preferred)” workflows to multiple MSBuild skills (incremental build, evaluation perf, parallelism, bin/obj clash detection, ResolveProjectReferences guidance).
  • Updated skill/agent frontmatter descriptions to mention binlog MCP usage and fallback behaviors.
  • Updated build-perf agent workflow to use MCP tools for top-down performance analysis, with a text-log replay alternative.
Show a summary per file
File Description
plugins/dotnet-msbuild/skills/resolve-project-references/SKILL.md Adds MCP-preferred guidance for task self-time analysis when ResolveProjectReferences looks expensive.
plugins/dotnet-msbuild/skills/incremental-build/SKILL.md Introduces an MCP-first workflow for diagnosing non-incremental rebuild causes.
plugins/dotnet-msbuild/skills/eval-performance/SKILL.md Adds an MCP-first workflow for evaluation timing/property/import analysis.
plugins/dotnet-msbuild/skills/check-bin-obj-clash/SKILL.md Reworks the primary workflow to use MCP tools to detect output/intermediate path clashes.
plugins/dotnet-msbuild/skills/build-parallelism/SKILL.md Adds MCP-first steps for identifying expensive projects/targets and target timing.
plugins/dotnet-msbuild/agents/build-perf.agent.md Switches the main top-down analysis flow to MCP tools, keeping a replay+grep alternate path.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

plugins/dotnet-msbuild/skills/resolve-project-references/SKILL.md:51

  • In the fallback command, -fl and -flp are formatted in a confusing way (-fl "-flp:..."). This reads like -fl is taking an argument and differs from the syntax used elsewhere in this plugin. Consider using the standard form (-fl followed by -flp:..., with PowerShell-appropriate quoting if needed) to avoid copy/paste failures.
```bash
dotnet msbuild build.binlog -noconlog -fl "-flp:v=diag;logfile=full.log;performancesummary"
grep "Task Performance Summary" -A 50 full.log
  • Files reviewed: 6/6 changed files
  • Comments generated: 2

Comment thread plugins/dotnet-msbuild/skills/check-bin-obj-clash/SKILL.md Outdated
Comment thread plugins/dotnet-msbuild/skills/eval-performance/SKILL.md Outdated
github-actions Bot added a commit that referenced this pull request May 22, 2026
github-actions Bot added a commit that referenced this pull request May 22, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Skill Validation Results

Skill Scenario Quality Skills Loaded Overfit Verdict
resolve-project-references Explain misleading ResolveProjectReferences time 3.3/5 → 5.0/5 🟢 ✅ resolve-project-references; tools: skill, glob / ✅ resolve-project-references; tools: skill ✅ 0.16
build-parallelism Analyze build parallelism bottlenecks 3.7/5 → 4.3/5 🟢 ✅ build-parallelism; tools: skill, binlog-binlog_overview, binlog-binlog_expensive_projects, binlog-binlog_projects, binlog-binlog_errors, glob, edit / ⚠️ NOT ACTIVATED ✅ 0.15 [1]
incremental-build Analyze incremental build issues 3.0/5 → 4.3/5 🟢 ✅ incremental-build; tools: skill, bash / ⚠️ NOT ACTIVATED ✅ 0.16 [2]
check-bin-obj-clash Diagnose bin/obj output path clashes 4.3/5 → 5.0/5 🟢 ✅ check-bin-obj-clash; tools: skill, glob, binlog-binlog_overview, binlog-binlog_projects, binlog-binlog_evaluations, binlog-binlog_double_writes, binlog-binlog_evaluation_properties, binlog-binlog_evaluation_global_properties / ✅ check-bin-obj-clash; tools: skill, binlog-binlog_overview, binlog-binlog_double_writes, binlog-binlog_evaluations, binlog-binlog_properties, binlog-binlog_evaluation_properties, binlog-binlog_diagnose ✅ 0.14 [3]
eval-performance Analyze MSBuild evaluation performance issues 5.0/5 → 4.7/5 🔴 ✅ eval-performance; tools: skill ✅ 0.20 [4]

[1] ⚠️ High run-to-run variance (CV=62%) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=88%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=515%) — consider re-running with --runs 5. (Isolated) Quality improved but weighted score is -17.2% due to: judgment, tokens (91656 → 276424), tool calls (12 → 27), time (75.8s → 101.9s)
[4] (Plugin) Quality unchanged but weighted score is -9.4% due to: quality, tokens (41594 → 73529)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 22, 2026 10:37
@JanKrivanek
Copy link
Copy Markdown
Member Author

/evaluate

@JanKrivanek JanKrivanek enabled auto-merge (squash) May 22, 2026 10:38
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 6/6 changed files
  • Comments generated: 6

Comment thread plugins/dotnet-msbuild/agents/build-perf.agent.md
Comment thread plugins/dotnet-msbuild/skills/build-parallelism/SKILL.md
Comment thread plugins/dotnet-msbuild/skills/incremental-build/SKILL.md
Comment thread plugins/dotnet-msbuild/skills/eval-performance/SKILL.md
Comment thread plugins/dotnet-msbuild/skills/check-bin-obj-clash/SKILL.md
Comment thread plugins/dotnet-msbuild/skills/resolve-project-references/SKILL.md
github-actions Bot added a commit that referenced this pull request May 22, 2026
github-actions Bot added a commit that referenced this pull request May 22, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Skill Validation Results

Skill Scenario Quality Skills Loaded Overfit Verdict
resolve-project-references Explain misleading ResolveProjectReferences time 3.3/5 → 5.0/5 🟢 ✅ resolve-project-references; tools: skill ✅ 0.12
build-parallelism Analyze build parallelism bottlenecks 3.7/5 → 3.7/5 ⏰ ✅ build-parallelism; tools: skill, binlog-binlog_overview, binlog-binlog_expensive_projects, binlog-binlog_errors, binlog-binlog_projects, edit, binlog-binlog_expensive_targets, glob / ⚠️ NOT ACTIVATED ✅ 0.20 [1]
incremental-build Analyze incremental build issues 3.0/5 → 4.7/5 🟢 ✅ incremental-build; tools: skill, bash / ⚠️ NOT ACTIVATED ✅ 0.14 [2]
check-bin-obj-clash Diagnose bin/obj output path clashes 3.7/5 → 5.0/5 🟢 ✅ check-bin-obj-clash; tools: glob, skill, binlog-binlog_overview, binlog-binlog_projects, binlog-binlog_double_writes, binlog-binlog_evaluations, binlog-binlog_evaluation_properties, binlog-binlog_evaluation_global_properties / ✅ check-bin-obj-clash; tools: glob, binlog-binlog_double_writes, binlog-binlog_diagnose, skill, binlog-binlog_evaluations, binlog-binlog_evaluation_properties, binlog-binlog_evaluation_global_properties, binlog-binlog_properties ✅ 0.14 [3]
eval-performance Analyze MSBuild evaluation performance issues 5.0/5 → 5.0/5 ✅ eval-performance; tools: skill ✅ 0.17 [4]

[1] ⚠️ High run-to-run variance (CV=432%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -7.1% due to: tokens (198097 → 381745), tool calls (15 → 25), time (82.4s → 107.6s)
[2] ⚠️ High run-to-run variance (CV=893%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=62%) — consider re-running with --runs 5. (Isolated) Quality improved but weighted score is -23.5% due to: judgment, tokens (102832 → 220260), quality, tool calls (12 → 23), time (68.7s → 95.4s)
[4] ⚠️ High run-to-run variance (CV=50%) — consider re-running with --runs 5

timeout — run(s) hit the (160s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

@JanKrivanek JanKrivanek merged commit 1a6e78c into main May 22, 2026
39 of 41 checks passed
@JanKrivanek JanKrivanek deleted the dev/jankrivanek/msbuild-skills-use-mcp branch May 22, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants