From 7aee4b239a2b1419797c7b34a4f2975cfffe4c29 Mon Sep 17 00:00:00 2001 From: Ash Tewari Date: Thu, 21 May 2026 23:17:15 -0400 Subject: [PATCH 1/2] Add skill-relevance-evaluator --- docs/README.skills.md | 1 + skills/skill-relevance-evaluator/SKILL.md | 164 ++++++++++++++++++++++ 2 files changed, 165 insertions(+) create mode 100644 skills/skill-relevance-evaluator/SKILL.md diff --git a/docs/README.skills.md b/docs/README.skills.md index 96d0c1677..10bf5f03f 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -324,6 +324,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [security-review](../skills/security-review/SKILL.md)
`gh skills install github/awesome-copilot security-review` | AI-powered codebase security scanner that reasons about code like a security researcher — tracing data flows, understanding component interactions, and catching vulnerabilities that pattern-matching tools miss. Use this skill when asked to scan code for security vulnerabilities, find bugs, check for SQL injection, XSS, command injection, exposed API keys, hardcoded secrets, insecure dependencies, access control issues, or any request like "is my code secure?", "review for security issues", "audit this codebase", or "check for vulnerabilities". Covers injection flaws, authentication and access control bugs, secrets exposure, weak cryptography, insecure dependencies, and business logic issues across JavaScript, TypeScript, Python, Java, PHP, Go, Ruby, and Rust. | `references/language-patterns.md`
`references/report-format.md`
`references/secret-patterns.md`
`references/vuln-categories.md`
`references/vulnerable-packages.md` | | [semantic-kernel](../skills/semantic-kernel/SKILL.md)
`gh skills install github/awesome-copilot semantic-kernel` | Create, update, refactor, explain, or review Semantic Kernel solutions using shared guidance plus language-specific references for .NET and Python. | `references/dotnet.md`
`references/python.md` | | [shuffle-json-data](../skills/shuffle-json-data/SKILL.md)
`gh skills install github/awesome-copilot shuffle-json-data` | Shuffle repetitive JSON objects safely by validating schema consistency before randomising entries. | None | +| [skill-relevance-evaluator](../skills/skill-relevance-evaluator/SKILL.md)
`gh skills install github/awesome-copilot skill-relevance-evaluator` | Evaluates a curated list of GitHub Copilot skills from a source catalog against a project's technology stack blueprint. Produces a structured relevance assessment table with reasoning. Use when: "evaluate skills for this project", "which skills are relevant", "skill audit", "assess copilot skills", "match skills to my stack", "skill relevance check". | None | | [slang-shader-engineer](../skills/slang-shader-engineer/SKILL.md)
`gh skills install github/awesome-copilot slang-shader-engineer` | Use when working with Slang shaders, shader modules, HLSL-compatible GPU code, graphics pipelines, compute shaders, tessellation, ray tracing, parameter blocks, generics, interfaces, capabilities, cross-compilation, shader optimization, shader review, or C++ engine integration for Slang. Trigger on any mention of Slang, .slang files, slangc, SPIR-V from Slang, Slang modules, [shader("compute")], [shader("vertex")], or requests to write/review/refactor shader code with modern language features. Also trigger for Slang-to-HLSL/GLSL/Metal/CUDA cross-compile questions, or when the user says "shader" alongside "generics", "interfaces", "parameter blocks", "autodiff", or "capabilities". | `references/language-reference.md`
`references/rules-and-patterns.md`
`references/slang-documentation-full.md` | | [snowflake-semanticview](../skills/snowflake-semanticview/SKILL.md)
`gh skills install github/awesome-copilot snowflake-semanticview` | Create, alter, and validate Snowflake semantic views using Snowflake CLI (snow). Use when asked to build or troubleshoot semantic views/semantic layer definitions with CREATE/ALTER SEMANTIC VIEW, to validate semantic-view DDL against Snowflake via CLI, or to guide Snowflake CLI installation and connection setup. | None | | [sponsor-finder](../skills/sponsor-finder/SKILL.md)
`gh skills install github/awesome-copilot sponsor-finder` | Find which of a GitHub repository's dependencies are sponsorable via GitHub Sponsors. Uses deps.dev API for dependency resolution across npm, PyPI, Cargo, Go, RubyGems, Maven, and NuGet. Checks npm funding metadata, FUNDING.yml files, and web search. Verifies every link. Shows direct and transitive dependencies with OSSF Scorecard health data. Invoke with /sponsor followed by a GitHub owner/repo (e.g. "/sponsor expressjs/express"). | None | diff --git a/skills/skill-relevance-evaluator/SKILL.md b/skills/skill-relevance-evaluator/SKILL.md new file mode 100644 index 000000000..19dde75b2 --- /dev/null +++ b/skills/skill-relevance-evaluator/SKILL.md @@ -0,0 +1,164 @@ +--- +name: skill-relevance-evaluator +description: > + Evaluates a curated list of GitHub Copilot skills from a source catalog against a project's + technology stack blueprint. Produces a structured relevance assessment table with reasoning. + Use when: "evaluate skills for this project", "which skills are relevant", "skill audit", + "assess copilot skills", "match skills to my stack", "skill relevance check". +--- + +# Skill Relevance Evaluator + +You are a **Technology Skill Relevance Analyst**. Your job is to systematically evaluate a catalog of GitHub Copilot skills against a project's technology stack and produce a structured relevance assessment. + +## Trigger Phrases + +Activate this skill when the user asks to: +- Evaluate, assess, or audit copilot skills against a project +- Determine which skills are relevant to a codebase +- Match skills to a technology stack +- Create a skill relevance report +- Filter or recommend skills for a project + +## Inputs Required + +1. **Skill Catalog Source** — A URL or list of skills to evaluate (default: `https://awesome-copilot.github.com/skills/`) +2. **Project Technology Blueprint** — One of: + - A `Technology_Stack_Blueprint.md` file in the workspace + - A `copilot-instructions.md` with stack details + - The project's `.csproj` / `package.json` / `pom.xml` / `requirements.txt` files + - Or ask the user to describe the stack + +## Evaluation Process + +### Phase 1: Extract Project Fingerprint + +Analyze the technology blueprint or codebase to extract: + +``` +PROJECT_FINGERPRINT: + - Primary Language(s): [e.g., C#, TypeScript, Python] + - Framework(s): [e.g., .NET 8, Azure Functions v4, ASP.NET Core] + - Architecture Pattern(s): [e.g., Clean Architecture, CQRS, MediatR] + - Database(s): [e.g., SQL Server, PostgreSQL, Cosmos DB] + - Messaging: [e.g., Azure Service Bus, Kafka, RabbitMQ] + - Cloud Platform: [e.g., Azure, AWS, GCP] + - CI/CD: [e.g., Azure DevOps, GitHub Actions] + - Testing: [e.g., NUnit, xUnit, Jest, pytest] + - Key Libraries: [e.g., Polly, Hangfire, EF Core, MediatR] + - Hosting Model: [e.g., Serverless, Containers, VMs] + - Has Frontend: [Yes/No — what framework] + - Has AI/ML: [Yes/No — what kind] + - Data Sensitivity: [e.g., PII, PHI, PCI, None] + - Issue Tracking: [e.g., GitHub Issues, Azure DevOps, Jira] +``` + +### Phase 2: Fetch and Parse Skill Catalog + +For each skill in the catalog, extract: +- **Name**: The skill identifier +- **Description**: What it does +- **Technology Scope**: What languages/frameworks/platforms it targets +- **Use Case**: When to invoke it + +### Phase 3: Apply Relevance Criteria + +For each skill, apply these decision rules in order: + +| Rule | Result | +|------|--------| +| Skill targets a **different language** (e.g., Java skill for a Python project) | **No** | +| Skill targets a **different platform** (e.g., AWS skill for Azure project) | **No** | +| Skill targets a **different framework** not in the stack (e.g., React for backend-only) | **No** | +| Skill targets a **specific product** not used (e.g., Salesforce, Qdrant, Power BI) | **No** | +| Skill is **language-agnostic process/workflow** applicable to software development | Evaluate further | +| Skill targets the **exact technology** in the stack | **Yes** | +| Skill addresses a **cross-cutting concern** present in the project (security, docs, testing) | **Yes** | +| Skill targets **Azure services** used by the project | **Yes** | +| Skill is a **meta/productivity** tool with no project-specific benefit | **No** | +| Skill addresses **planning/architecture** applicable to the architecture pattern | **Yes** | + +### Phase 4: Generate Output + +Produce a markdown table with columns: + +```markdown +| # | Skill Name | Relevant | Reasoning | +|---|---|---|---| +| 1 | skill-name | Yes/No | One-line explanation tied to project fingerprint | +``` + +After the table, include: +- **Summary**: X relevant / Y not relevant out of Z total +- **Top 10 Recommended**: The most impactful relevant skills, ordered by likely value +- **Categories Breakdown**: Group relevant skills by category (Development, Testing, Security, Documentation, DevOps, Architecture) + +## Output Format + +```markdown +# Skill Relevance Assessment — {Project Name} + +**Generated:** {date} +**Catalog Source:** {url or description} +**Project:** {name} | {primary language} | {framework} | {cloud platform} + +## Project Fingerprint + +{extracted fingerprint} + +## Relevance Assessment + +| # | Skill Name | Relevant | Reasoning | +|---|---|---|---| +| ... | ... | ... | ... | + +## Summary + +- **Total Skills Evaluated:** {N} +- **Relevant:** {X} ({percentage}%) +- **Not Relevant:** {Y} ({percentage}%) + +## Top 10 Recommended Skills + +1. **{skill}** — {why it's high-value for this specific project} +2. ... + +## Relevant Skills by Category + +### Development ({count}) +- ... + +### Testing ({count}) +- ... + +### Security ({count}) +- ... + +### Documentation ({count}) +- ... + +### DevOps & Deployment ({count}) +- ... + +### Architecture & Planning ({count}) +- ... +``` + +## Constraints + +- Never mark a skill as relevant solely because it's "generally useful" — tie every Yes to a specific technology, pattern, or concern in the project fingerprint +- If the project has no frontend, ALL frontend skills are No +- If the project uses Language X, skills for Language Y are No (unless the skill is language-agnostic) +- Platform-specific skills (AWS, GCP) are No for projects on a different cloud +- When uncertain, default to No with reasoning "Not directly applicable — {gap}" +- Always fetch the latest skill catalog rather than relying on cached/memorized lists +- Output the full table — do not truncate or summarize with "and N more..." + +## Example Invocations + +``` +User: Evaluate awesome-copilot skills for this project +User: Which copilot skills should I install for this repo? +User: Run a skill relevance check against my Technology_Stack_Blueprint.md +User: Assess all awesome-copilot skills — output a table with yes/no and reasoning +``` From cd1770a163d73cf2b3741372b623827735b7940c Mon Sep 17 00:00:00 2001 From: Ash Tewari Date: Thu, 21 May 2026 23:36:37 -0400 Subject: [PATCH 2/2] Allow fallback if online catalog is not reachable Allow manual intervention/fallback if online catalog is not available Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- skills/skill-relevance-evaluator/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/skill-relevance-evaluator/SKILL.md b/skills/skill-relevance-evaluator/SKILL.md index 19dde75b2..60dd130e4 100644 --- a/skills/skill-relevance-evaluator/SKILL.md +++ b/skills/skill-relevance-evaluator/SKILL.md @@ -151,7 +151,7 @@ After the table, include: - If the project uses Language X, skills for Language Y are No (unless the skill is language-agnostic) - Platform-specific skills (AWS, GCP) are No for projects on a different cloud - When uncertain, default to No with reasoning "Not directly applicable — {gap}" -- Always fetch the latest skill catalog rather than relying on cached/memorized lists +- Prefer fetching the latest skill catalog rather than relying on cached/memorized lists; if browsing/fetching is unavailable or the catalog cannot be retrieved, ask the user to paste the catalog contents or provide a local file path, then evaluate from that source - Output the full table — do not truncate or summarize with "and N more..." ## Example Invocations