Skip to content

Add insights into the infra planner skill#2345

Open
micha31r wants to merge 15 commits into
microsoft:mainfrom
micha31r:user/mren/insights-integration
Open

Add insights into the infra planner skill#2345
micha31r wants to merge 15 commits into
microsoft:mainfrom
micha31r:user/mren/insights-integration

Conversation

@micha31r
Copy link
Copy Markdown
Contributor

@micha31r micha31r commented May 21, 2026

Description

This PR concerns the azure-enterprise-infra-planner skill. It integrates insights into the infrastructure planning process, which are patterns derived from a user’s existing Azure environment (generated using the insights_get MCP tool). This helps align plans more closely with how users already use Azure, with the goal of reducing manual editing and increasing plan acceptance.

This PR contains two main changes:

  1. 568443e - Refactor current phases into 6 distinct files.
  2. 59fdb03 - Integrate insights into the phases.

Note: This PR depends on the azure-insights tool PR in Azure MCP being merged first. Reviews are still appreciated so this PR can be merged as soon as the dependency is ready.

Testing and Metrics

  • No changes were made to the frontmatter and trigger phrases - skill invocation remains the same.
  • The updated skill was tested against our custom rubric to assess the infra plan quality - there is no regressions on the four core dimensions (Goal Alignment, WAF Conformance, Dependency Completeness, and Deployment Viability). Scores ~4.5 on 0-5 scale.
  • Compared to baseline plans, insights plans have about 18% higher property alignment.

Checklist

  • Tests pass locally (cd tests && npm test)
  • If modifying skill descriptions: verified routing correctness with integration tests (npm run test:skills:integration -- <skill>)
  • If modifying skill USE FOR / DO NOT USE FOR / PREFER OVER clauses: confirmed no routing regressions for competing skills

@micha31r micha31r force-pushed the user/mren/insights-integration branch from 00046ff to 86d8c08 Compare May 21, 2026 12:06
@micha31r micha31r changed the title User/mren/insights integration Add insights into the infra planner skill May 21, 2026
@micha31r micha31r force-pushed the user/mren/insights-integration branch 2 times, most recently from 1dee28d to 8c93843 Compare May 21, 2026 15:03
@micha31r micha31r force-pushed the user/mren/insights-integration branch from 7683113 to 787dd22 Compare May 21, 2026 15:24
@micha31r micha31r marked this pull request as ready for review May 21, 2026 15:25
@micha31r micha31r requested a review from Jbrocket as a code owner May 21, 2026 15:25
Copilot AI review requested due to automatic review settings May 21, 2026 15:25
@micha31r micha31r requested a review from jongio as a code owner May 21, 2026 15:25
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the azure-enterprise-infra-planner skill to incorporate environment-derived “insights” (via the insights_get MCP tool) into the infra planning workflow, while also refactoring the workflow into phase-specific reference files.

Changes:

  • Refactors the planner workflow into seven phase documents and updates the main workflow entrypoint to reference them.
  • Introduces insights extraction/application guidance and adds plan schema + integration test expectations for inputs.insightsApplied.
  • Updates skill docs (quick reference, checklists, verification steps) to reflect the new insights-aware process.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
tests/azure-enterprise-infra-planner/integration.test.ts Adds soft-checking for insights generation and asserts inputs.insightsApplied exists.
plugin/skills/azure-enterprise-infra-planner/version.json Bumps skill version config from 1.1 to 1.2.
plugin/skills/azure-enterprise-infra-planner/SKILL.md Updates tool list, links to workflow/phases, and adds sample plan link.
plugin/skills/azure-enterprise-infra-planner/references/workflow.md Replaces inline workflow with a phased table and adds insights artifact output.
plugin/skills/azure-enterprise-infra-planner/references/waf-checklist.md Rewords checklist guidance to be more principle-based and explicit about tradeoffs.
plugin/skills/azure-enterprise-infra-planner/references/verification.md Adds an “Insights Checks” section to verification.
plugin/skills/azure-enterprise-infra-planner/references/resources/README.md Updates phase reference for when resource reference files should be loaded.
plugin/skills/azure-enterprise-infra-planner/references/constraints/README.md Updates phase reference for when constraint files should be loaded.
plugin/skills/azure-enterprise-infra-planner/references/research.md Removes the old monolithic research phase document (superseded by phased docs).
plugin/skills/azure-enterprise-infra-planner/references/plan-schema.md Adds required inputs.insightsApplied and updates references to new phase docs.
plugin/skills/azure-enterprise-infra-planner/references/phases/1-extract-insights.md New Phase 1 doc: extract and persist insights to .azure/insights.json.
plugin/skills/azure-enterprise-infra-planner/references/phases/2-research-best-practices.md New Phase 2 doc: clarify requirements, derive sub-goals, run WAF MCP research.
plugin/skills/azure-enterprise-infra-planner/references/phases/3-research-resources.md New Phase 3 doc: apply WAF checklist, integrate insights, and do resource lookups.
plugin/skills/azure-enterprise-infra-planner/references/phases/4-generate-plan.md New Phase 4 doc: write the plan JSON and set status to draft.
plugin/skills/azure-enterprise-infra-planner/references/phases/5-verify.md New Phase 5 doc: verify plan, fix issues, and manage approval loop.
plugin/skills/azure-enterprise-infra-planner/references/phases/6-generate-iac.md New Phase 6 doc: generate Bicep/Terraform from approved plan.
plugin/skills/azure-enterprise-infra-planner/references/phases/7-deploy.md New Phase 7 doc: deploy with confirmation and correct scope selection.

Comment thread tests/azure-enterprise-infra-planner/integration.test.ts Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/references/workflow.md Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/references/phases/6-generate-iac.md Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/references/phases/7-deploy.md Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/references/resources/README.md Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/references/constraints/README.md Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/references/plan-schema.md Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/references/verification.md Outdated
Comment thread plugin/skills/azure-enterprise-infra-planner/SKILL.md Outdated
Comment thread tests/azure-enterprise-infra-planner/integration.test.ts Outdated
Copy link
Copy Markdown
Member

@JasonYeMSFT JasonYeMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since your new skill instructions asked the agent to use sub agent, I would like to learn more about how the agent reacts in your testing. For VS Code and Copilot CLI, is the agent able to create ad-hoc subagents to execute the steps you described?

@micha31r micha31r requested a review from RickWinter as a code owner May 22, 2026 02:07
Michael Ren and others added 2 commits May 22, 2026 12:09
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@micha31r micha31r force-pushed the user/mren/insights-integration branch from bfdfa88 to ed53e90 Compare May 22, 2026 02:20
@micha31r
Copy link
Copy Markdown
Contributor Author

micha31r commented May 25, 2026

Since your new skill instructions asked the agent to use sub agent, I would like to learn more about how the agent reacts in your testing. For VS Code and Copilot CLI, is the agent able to create ad-hoc subagents to execute the steps you described?

This works really well. We tested on several models including Sonnet 4.6, GPT 5.5. Opus 4.6, Opus 4.7 and in all of the tests it was able to spawn the subagents and run them in parallel when instructed to. It also spawns the right kind of agents as well. For example, we can explicitly tell it to spawn general-purpose subagents.

@micha31r micha31r requested a review from JasonYeMSFT May 25, 2026 13:04
Comment thread plugin/skills/azure-enterprise-infra-planner/version.json
@JasonYeMSFT
Copy link
Copy Markdown
Member

@Jbrocket Could you please review this PR as a code owner?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants