A comprehensive fix for the OpenRouter integration that adds AI Task entity support with vision capabilities for camera snapshot analysis and structured data generation.
- 🤖 AI Task Entity Support: Full
ai_task.generate_dataservice integration - 📷 Camera Snapshot Analysis: Direct camera feed processing with vision models
- 🔧 OpenAI Library Compatibility: Works with all OpenAI library versions (1.x and 2.x)
- 📊 Structured Data Output: JSON responses for automation workflows
- 💰 Free Model Support: Access to Qwen2.5-VL-72B and other free vision models
- ✅ Home Assistant 2024.12+: Compatible with latest HA versions
- 🌍 200+ Models: Access to all OpenRouter models including vision capabilities
- Install HACS if you haven't already
- Add this repository as a custom repository:
- Go to HACS → Integrations
- Click the three dots menu → Custom repositories
- Add:
https://github.com/brouwerict/open_router_fixed - Category: Integration
- Search for "OpenRouter Fixed" and install it
- Restart Home Assistant
- Copy the
custom_components/openrouter_fixedfolder to your Home Assistantcustom_componentsdirectory - Restart Home Assistant
- Go to Settings → Devices & Services
- Click "+ Add Integration"
- Search for "OpenRouter Fixed"
- Enter your OpenRouter API key (get one at OpenRouter.ai)
- Configure AI Task entities and/or Conversation agents
This integration fully supports image analysis through AI Tasks, perfect for security, vehicle detection, and home monitoring.
qwen/qwen2.5-vl-72b-instruct:free- Excellent 72B vision modelqwen/qwen2.5-vl-32b-instruct:free- Good balance of speed/quality
anthropic/claude-3-haiku- Fast & cost-effective ($0.25/1M tokens)anthropic/claude-3-5-sonnet- High accuracy visionopenai/gpt-4o-mini- OpenAI's efficient vision modelopenai/gpt-4o- Top-tier OpenAI visiongoogle/gemini-pro-vision- Google's vision model
# Vehicle detection automation
automation:
- alias: "Detect Vehicles on Driveway"
trigger:
- platform: state
entity_id: binary_sensor.driveway_motion
to: 'on'
action:
- service: ai_task.generate_data
data:
entity_id: ai_task.qwen_qwen2_5_vl_72b_instruct # Free model!
task_name: "Vehicle Detection"
instructions: |
Analyze this driveway camera image.
Count all vehicles and return JSON:
{
"total_vehicles": 0,
"total_cars": 0,
"total_motorcycles": 0,
"vehicle_list": "Details of detected vehicles"
}
attachments:
media_content_id: media-source://camera/driveway_camera
media_content_type: image/jpeg
response_variable: vehicle_data
# Update counters
- service: input_number.set_value
target:
entity_id: input_number.car_count
data:
value: "{{ (vehicle_data.data | from_json).total_cars }}"# Security monitoring with structured data
- service: ai_task.generate_data
data:
entity_id: ai_task.anthropic_claude_3_haiku
task_name: "Security Analysis"
instructions: "Analyze security camera for threats"
structure:
person_detected:
description: "Whether a person is visible"
selector:
boolean:
required: true
person_count:
description: "Number of people detected"
selector:
number:
min: 0
max: 10
required: true
threat_level:
description: "Security threat assessment"
selector:
select:
options:
- "none"
- "low"
- "medium"
- "high"
required: true
description:
description: "Detailed description of the scene"
selector:
text:
required: true
attachments:
media_content_id: media-source://camera/front_door
media_content_type: image/jpeg
response_variable: security_resultservice: ai_task.generate_data
data:
entity_id: ai_task.model_name # Your AI Task entity
task_name: "Task Description" # Required field
instructions: "What to analyze" # Your prompt
attachments:
media_content_id: media-source://camera/camera_name # Camera source
media_content_type: image/jpeg # Media type
structure: # Optional: Define output structure
field_name:
description: "Field description"
selector:
type: # boolean, number, text, etc.
required: true/false
response_variable: result # Variable to store responseProblem: ImportError: cannot import name 'ResponseFormatJSONSchema'
Solution: Dynamic import system with fallbacks for all OpenAI library versions
Problem: No AI Task entities available in original integration
Solution: Complete AITaskEntity implementation with attachment support
Problem: Vision models filtered out or unavailable
Solution: Removed restrictive model filtering, all 200+ models now accessible
Problem: Cannot process camera snapshots or images
Solution: Multi-format attachment handler supporting:
- Camera media sources
- File paths
- Base64 encoded images
- Binary data
Problem: Inconsistent JSON parsing failures
Solution: Robust parsing with graceful fallback to raw text
| Model | Cost per 1000 calls | Quality | Speed |
|---|---|---|---|
| Qwen2.5-VL-72B | $0.00 (FREE!) | Excellent | Good |
| Claude 3 Haiku | $0.61 | Good | Fast |
| GPT-4o-mini | $0.42 | Good | Fast |
| Claude 3.5 Sonnet | $7.80 | Excellent | Medium |
Recommendation: Use qwen/qwen2.5-vl-72b-instruct:free for production - it's completely free and performs excellently!
- Detect packages at front door
- Notify when delivery arrives
- Log delivery times- Count cars in driveway
- Detect unknown vehicles
- Track parking occupancy- Person detection
- Threat assessment
- Anomaly detection- Check if pets are inside/outside
- Food bowl monitoring
- Activity tracking- Check entity_id exists in Developer Tools > States
- Verify model name is correct
- Ensure API key has sufficient credits
- Add "Return ONLY valid JSON" to instructions
- Use simpler JSON structure
- Try different model (Qwen models are very consistent)
- Use
media-source://camera/camera_nameformat - Ensure camera entity is available
- Check file permissions for file attachments
- Visit OpenRouter.ai
- Create account
- Generate API key
- Add credits (or use free models!)
For issues or questions, please open an issue on GitHub.
Pull requests welcome! See TECHNICAL_PR_DESCRIPTION.md for technical details.
Apache 2.0 - Same as Home Assistant
- Based on the official OpenRouter integration for Home Assistant
- Enhanced with AI Task support and vision capabilities
- Original author: @joostlek