Turn your iPhone into an AI agent host. AgentOS is a native iOS app that runs Claude AI with full tool-use capabilities — your phone can see, hear, speak, browse, code, trade, and communicate autonomously.
AgentOS transforms an iPhone into a powerful AI agent that can:
- See — Camera, photo library, computer vision, AR scene analysis
- Hear & Speak — On-device STT (Whisper), TTS (ElevenLabs), voice channel
- Browse — Web search, fetch pages, cloud browser automation (Browserbase)
- Code — GitHub repos, branches, PRs, issues, file editing (10 tools)
- Trade — Taiwan stocks, US equities, and cryptocurrency markets (11 tools)
- Communicate — 10 channels: Telegram, Discord, LINE, WhatsApp, Slack, Teams, Matrix, Signal, Voice, built-in chat
- Remember — Basic + semantic vector memory with cross-session persistence
- Automate — Scheduled tasks (cron), background execution, multi-agent discussions
- Create — DALL-E image generation, HeyGen video, Canva design
- Manage — Google Drive, Gmail, Calendar, Contacts, Photos, Reminders, HomeKit, Health
Built with pure Swift calling the Claude Messages API directly — no subprocess spawning, no bridging frameworks.
| Category | Count | Details |
|---|---|---|
| Native Tools | 165+ | 59 tool files across hardware, cloud, social, coding, market data |
| AI Agents | 30 | Specialized personas with dedicated dashboard UIs |
| Smart Skills | 34 | Lightweight composable capabilities |
| Channels | 10 | Built-in, Telegram, Discord, LINE, WhatsApp, Slack, Teams, Matrix, Signal, Voice |
| AI Providers | 5 | Anthropic, z.ai, OpenRouter, Gemini, MLX Local (on-device) |
| MCP Servers | Dynamic | Streamable HTTP transport (iOS-native) |
| Languages | 3 | English, 繁體中文, 日本語 (live-switch) |
| Siri Shortcuts | 11 | 6 voice-activated + 5 Shortcuts-only intents |
| Swift Files | 232+ | ~76K LOC |
30 pre-built agents, each with a dedicated dashboard UI:
| Agent | What It Does |
|---|---|
| Personal Assistant | General assistant — web search, notes, journal, memories |
| GitHub Agent | Full coding workflow: planner → coding → reviewer pipeline |
| Market Agent | Taiwan/US stocks + crypto with real-time data |
| Threads Publisher | Meta Threads: search → write → analyze (14 tools) |
| Schedule Manager | Calendar, reminders, and scheduled task automation |
| Photo Manager | Photo library management with ML-powered search |
| Automation Agent | Cron jobs, webhooks, multi-step automation |
| Cloud Dev Agent | Browser + code execution + image gen in the cloud |
| Channel Hub | Multi-channel monitoring and response |
| + 21 more | Including hardware agents (BLE, NFC, AR, HomeKit, Health, Music, Navigation) |
Create your own agents with the 5-step Agent Creation Wizard or import community agents via .agentpkg packages.
34 composable skills that enhance any agent — no dedicated UI needed:
| Skill | Description |
|---|---|
| Market Analyst | Taiwan stock market analysis |
| US Stock Analyst | US equity market data (Polygon) |
| Crypto Analyst | Cryptocurrency market analysis |
| Threads Publisher | Meta Threads social media (14 tools) |
| Coding | GitHub coding workflow (10 tools) |
| Brainstormer | Creative ideation and brainstorming |
| Planner | Project and task planning |
| Reviewer | Code and content review |
| Translator | Multi-language translation |
| Document Processor | Document analysis and processing |
| + 24 more | Covering SEO, email, cloud infra, security scanning, and more |
| Channel | Foreground | Background (Always-On) |
|---|---|---|
| Built-in Chat | Native SwiftUI | Always available |
| Telegram | Long polling | APNs relay |
| Discord | WebSocket Gateway | APNs relay |
| LINE | Webhook | APNs relay |
| Webhook | APNs relay | |
| Slack | WebSocket | APNs relay |
| Teams | Webhook | APNs relay |
| Matrix | Sync API | APNs relay |
| Signal | Webhook | APNs relay |
| Voice | On-device STT/TTS | — |
Always-On Architecture: Cloudflare Worker relay enables background message processing via APNs push notifications, even when the app is suspended.
| Provider | Model | Use Case |
|---|---|---|
| Anthropic | Claude Opus 4 | Primary (adaptive thinking, tool_use) |
| z.ai | Claude via z.ai | Alternative endpoint |
| OpenRouter | Multiple models | Model diversity |
| Gemini | Gemini Pro | Google integration |
| MLX Local | Qwen3, Gemma, Llama | On-device inference (no API key) |
Auto-failover between providers when one is unavailable.
Extend AgentOS with any Model Context Protocol server using Streamable HTTP transport — the only MCP transport that works on iOS (no subprocess spawning).
Pre-configured servers included. Add your own via the in-app MCP settings.
┌──────────────────────────────────────────────────┐
│ AgentOS (iOS App) │
├───────────┬───────────┬────────────┬─────────────┤
│ Channel │ Agent │ Tool │ MCP │
│ Manager │ Engine │ Dispatch │ Client │
│ (10 ch) │ (core) │ (165+) │ (HTTP) │
├───────────┴───────────┴────────────┴─────────────┤
│ Claude Messages API (agentic tool_use loop) │
├──────────────────────────────────────────────────┤
│ 5 AI Providers │ Always-On Relay (CF Worker) │
├──────────────────┴───────────────────────────────┤
│ iOS Hardware: Camera, GPS, Audio, BLE, NFC, AR │
└──────────────────────────────────────────────────┘
Agentic Loop: Message → Claude API → tool_use? → execute tools → append results → loop until end_turn. Safety limit: 50 tool calls per turn.
| Hook | Purpose |
|---|---|
| ActivityLogHook | Logs all tool usage for audit trail |
| PrivacyHook | Requires explicit permission for sensitive operations (Drive, Gmail, GitHub writes, contacts, BLE, HomeKit) |
| ConstitutionHook | Constitutional AI guardrails |
- All API keys stored in iOS Keychain (never in plain text, never in UserDefaults)
- Channel allow-lists: only explicitly approved chats can send commands (fail-closed)
- MCP connections over HTTPS only
- No data collection — everything stays on your device
- Sensitive operations gated by PrivacyHook (requires user approval)
- Rate limiting via token-bucket algorithm
- Telegram/Discord webhook verification (Ed25519, HMAC-SHA256)
- iPhone running iOS 16.0 or later
- At least one AI provider API key:
- Anthropic (recommended)
- z.ai
- OpenRouter
- Or use MLX Local for fully on-device inference (no API key needed)
- Optional: Telegram Bot Token, Discord Bot Token, LINE Channel Token, etc.
Found a bug or have a feature request? Please open an issue.
- Bug reports: Use the bug report template — include iOS version, steps to reproduce, and expected vs actual behavior
- Feature requests: Use the feature request template — describe your use case
See CONTRIBUTING.md for guidelines.
For security vulnerabilities, please see SECURITY.md. Do not open public issues for security vulnerabilities.
- Website: agent.lucas-futures.com
- Issues: github.com/claude-world/agentOS/issues
MIT License — see LICENSE for details.
Built with Claude by SORAING CO., LTD.
