hybrid-inference

Here are 6 public repositories matching this topic...

brontoguana / krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

transformer inference-engine inference-optimization mixture-of-experts cpu-inference large-language-models gpu-inference llm-inference high-performance-inference hybrid-inference gguf-model-support llama-cpp-alternative

Updated Apr 18, 2026
C++

mhayes853 / swift-cactus

Star

Crossplatform Swift SDK for Cactus hybrid inference and running LLMs locally in your app.

swift ai json-schema edge swift-android llm llms llm-inference hybrid-inference

Updated Apr 18, 2026
Swift

LiAgent OS is a local-first AI agent OS for building a private personal assistant with governed autonomy across local models and hybrid cloud services. It brings conversation, tool use, multi-agent orchestration, task scheduling, heartbeat execution, human approval, and auditability into one long-lived loop.

websocket discord-bot multi-agent observability mlx human-in-the-loop ai-agents local-models fastapi tool-use task-scheduling workflow-orchestration local-first llm model-context-protocol hybrid-inference agent-runtime governed-ai

Updated Mar 9, 2026
Python

BjornMelin / local-llm-workbench

Star

🧠 A comprehensive toolkit for benchmarking, optimizing, and deploying local Large Language Models. Includes performance testing tools, optimized configurations for CPU/GPU/hybrid setups, and detailed guides to maximize LLM performance on your hardware.