Skip to content

feat(microsoft-foundry): add Tracing Insights API skill for automated anomaly detection#2276

Merged
jongio merged 2 commits into
microsoft:mainfrom
imatiach-msft:feat/tracing-insights-api-only
May 16, 2026
Merged

feat(microsoft-foundry): add Tracing Insights API skill for automated anomaly detection#2276
jongio merged 2 commits into
microsoft:mainfrom
imatiach-msft:feat/tracing-insights-api-only

Conversation

@imatiach-msft
Copy link
Copy Markdown
Contributor

@imatiach-msft imatiach-msft commented May 15, 2026

Summary

Add the Tracing Insights API skill file for automated anomaly detection on agent evaluation scores. This is a subset of PR #2155 -- extracted to unblock merge while FAOS optimization integration is completed separately.

What's included

  • New file: plugin/skills/microsoft-foundry/foundry-agent/trace/references/tracing-insights-api.md
    • v1-beta2 API endpoint, auth, query parameters
    • Response schema with agents[] grouping and relatedSpans containing operationId
    • KQL template for querying full trace content from App Insights
    • Changepoint detection explanation
  • Modified: trace/trace.md -- added entry point for "auto-detect agent issues" user intent

What's NOT included (deferred to follow-up PR)

  • FAOS optimization skill (faos-optimization.md) -- FAOS API not yet GA
  • E2E insights-to-optimize loop (insights-to-optimize.md) -- depends on FAOS
  • SKILL.md description changes -- not needed for this subset, avoids snapshot churn

Validated

  • Tracing Insights API v1-beta2 tested across 3 regions (eastus2euap, westcentralus, ukwest)
  • Returns real insights with relatedSpans containing operationId that maps to App Insights trace IDs
  • No SKILL.md changes = no snapshot updates needed

Copilot AI review requested due to automatic review settings May 15, 2026 15:26
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Microsoft Foundry trace reference for using the Tracing Insights API to detect agent quality anomalies, and wires it into the trace sub-skill entry points.

Changes:

  • Added tracing-insights-api.md with endpoint, auth, request/response schema, KQL follow-up query, and changepoint guidance.
  • Added a trace entry point for automated insight/anomaly detection.
  • Accidentally changed an existing heading anchor in trace.md.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md Adds the Tracing Insights API entry point and modifies the App Insights setup heading.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/tracing-insights-api.md New reference documenting the Tracing Insights API workflow.

Comment thread plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md Outdated
… anomaly detection

Add skill files and reference documentation for the Tracing Insights API,
enabling Copilot to retrieve automated anomaly-detection insights from
Azure AI Foundry agent traces.

- trace.md: skill descriptor with triggers, instructions, and examples
- tracing-insights-api.md: API reference (endpoint, auth, parameters, response)
- Region-agnostic API — any region endpoint works with any project

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@imatiach-msft imatiach-msft force-pushed the feat/tracing-insights-api-only branch from ae1022c to 148e980 Compare May 15, 2026 15:45
@imatiach-msft
Copy link
Copy Markdown
Contributor Author

@jongio @XOEEst gentle ping on this PR

tmeschter
tmeschter previously approved these changes May 15, 2026
@imatiach-msft
Copy link
Copy Markdown
Contributor Author

@XOEEst, @XiaofuHuang, @anchenyi, @ankitbko, @jongio, and/or @tendau might you be able to review this PR? I need an approval from one of the code owners to be able to merge. Thank you!

Copy link
Copy Markdown
Collaborator

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-scoped docs addition that follows the existing reference structure. One note on region guidance.

Address review feedback: lead with region-agnostic behavior, note
same-region is preferred for latency, use eastus2 instead of
eastus2euap as the default.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@imatiach-msft
Copy link
Copy Markdown
Contributor Author

imatiach-msft commented May 15, 2026

gentle ping @XOEEst, @XiaofuHuang, @anchenyi, @ankitbko, @jongio, and/or @tendau . I need an approval from one of the code owners to be able to merge. I've addressed all comments in current PR. Thank you!

Copy link
Copy Markdown
Collaborator

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addresses my previous feedback - region guidance now leads with the agnostic behavior and defaults to eastus2. Clean fix.

@jongio jongio merged commit ae584b5 into microsoft:main May 16, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants