Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Foundry Local — AI Coding Assistant Context

Foundry Local is an on-device AI inference runtime. It provides:

- **Chat completions** (text generation) via native SDK or OpenAI-compatible REST API
- **Audio transcription** (speech-to-text via Whisper) via native SDK
- **Automatic hardware acceleration** — NPU > GPU > CPU, zero detection code needed

## SDK Quick Reference

### JavaScript (`foundry-local-sdk` on npm)

```js
import { FoundryLocalManager } from 'foundry-local-sdk';
const manager = FoundryLocalManager.create({ appName: 'foundry_local_samples' });

// Chat
const chatModel = await manager.catalog.getModel('qwen2.5-0.5b');
await chatModel.download();
await chatModel.load();
const chatClient = chatModel.createChatClient();
const response = await chatClient.completeChat([
{ role: 'user', content: 'Hello' }
]);

// Audio transcription
const whisperModel = await manager.catalog.getModel('whisper-tiny');
await whisperModel.download();
await whisperModel.load();
const audioClient = whisperModel.createAudioClient();
const result = await audioClient.transcribe('recording.wav');
```

### C# (.NET — `Microsoft.AI.Foundry.Local` NuGet)

```csharp
using Microsoft.AI.Foundry.Local;

var config = new Configuration { AppName = "foundry_local_samples" };
await FoundryLocalManager.CreateAsync(config);
Comment on lines +38 to +40
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The C# quick reference won't compile as written: FoundryLocalManager.CreateAsync requires an ILogger parameter (and optional cancellation token). Update the snippet to pass a logger (e.g., NullLogger.Instance or a real ILogger), matching the SDK's actual signature.

Suggested change
var config = new Configuration { AppName = "foundry_local_samples" };
await FoundryLocalManager.CreateAsync(config);
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
var config = new Configuration { AppName = "foundry_local_samples" };
await FoundryLocalManager.CreateAsync(config, NullLogger.Instance);

Copilot uses AI. Check for mistakes.
var mgr = FoundryLocalManager.Instance;
var catalog = await mgr.GetCatalogAsync();

// Chat
var chatModel = await catalog.GetModelAsync("qwen2.5-0.5b")
?? throw new Exception("Model not found");
await chatModel.DownloadAsync();
await chatModel.LoadAsync();
var chatClient = await chatModel.GetChatClientAsync();

// Audio transcription
var whisperModel = await catalog.GetModelAsync("whisper-tiny")
?? throw new Exception("Model not found");
await whisperModel.DownloadAsync();
await whisperModel.LoadAsync();
var audioClient = await whisperModel.GetAudioClientAsync();
```

## Key Rules

- **Never hardcode ports.** The native SDK runs inference in-process — no port needed for chat or audio. If you use the optional REST web server, read the port from the manager after starting it (JS: `manager.urls` after `startWebService()`, C#: `FoundryLocalManager.Instance.Urls` after `StartWebServiceAsync()`).
- **Use model aliases**, not full model IDs. Aliases like `qwen2.5-0.5b` and `whisper-tiny` auto-select the best variant for the user's hardware.
- **One manager handles everything.** Don't create separate runtimes for chat and audio.
- **Do NOT use `whisper.cpp`, `llama.cpp`, `@huggingface/transformers`, or `ollama`** alongside Foundry Local — it handles all of these use cases.

## Model Aliases

| Task | Aliases |
|------|---------|
| Chat | `phi-3.5-mini`, `phi-4-mini`, `qwen2.5-0.5b`, `qwen2.5-coder-0.5b` |
| Audio Transcription | `whisper-tiny`, `whisper-base`, `whisper-small` |
Comment on lines +68 to +71
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The markdown table under "Model Aliases" is malformed (rows start with ||), which will render as an extra empty column or not render as a table depending on the markdown renderer. Update the table to standard |-delimited syntax.

Copilot uses AI. Check for mistakes.
Comment on lines +70 to +71
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several aliases listed here (e.g., phi-4-mini, qwen2.5-coder-0.5b, whisper-base, whisper-small) are not referenced anywhere else in this repo’s SDK docs/examples/tests, while phi-3.5-mini, qwen2.5-0.5b, and whisper-tiny are. To avoid teaching assistants nonexistent aliases, please confirm these names are actually present in the Foundry Local catalog and, if not, remove or replace them with aliases that are known to exist.

Suggested change
| Chat | `phi-3.5-mini`, `phi-4-mini`, `qwen2.5-0.5b`, `qwen2.5-coder-0.5b` |
| Audio Transcription | `whisper-tiny`, `whisper-base`, `whisper-small` |
| Chat | `phi-3.5-mini`, `qwen2.5-0.5b` |
| Audio Transcription | `whisper-tiny` |

Copilot uses AI. Check for mistakes.