Stars
Forks
Watchers
Developer links
LocalAI
With over 48,000 GitHub stars and monthly releases since March 2023, LocalAI is the self-hosted AI engine that replaces every OpenAI endpoint with a single Docker container running on your own infrastructure — serving chat completions, image generation, text-to-speech, speech-to-text, embeddings, vision, video generation, and function calling through identical API schemas that require zero application code changes. The composable backend architecture isolates each inference engine as a separate gRPC service running in its own OCI container, so llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, Stable Diffusion, and Flux install on demand without touching the core, can run on separate machines, and a fault in one never affects others. Hardware acceleration spans NVIDIA CUDA 12 and 13, AMD ROCm, Intel oneAPI/SYCL, Apple Silicon Metal, Vulkan, and NVIDIA Jetson L4T — or runs entirely on CPU without any GPU. Built-in AI agents support autonomous tool use, retrieval-augmented generation, Model Context Protocol integration, and skill-based workflows directly in the web interface. The model gallery provides curated YAML configuration files for hundreds of models that install with a single command, while P2P federated inference distributes model shards across multiple machines for running models larger than any single node's memory. Multi-user API key authentication with quotas and role-based access enables team deployments. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.
Benefits
- Drop-In OpenAI API Replacement
- Identical endpoint schemas for chat completions, images, audio, embeddings, and function calling mean existing applications point to LocalAI without any code changes — swap cloud for self-hosted instantly.
- 60+ Composable Inference Backends
- Each backend runs as an isolated gRPC service installed on demand — llama.cpp, vLLM, SGLang, whisper.cpp, diffusers, and MLX operate independently with zero cross-contamination and individual upgrade cycles.
- Universal Hardware Acceleration
- NVIDIA CUDA, AMD ROCm, Intel oneAPI, Apple Silicon Metal, Vulkan, and NVIDIA Jetson support means any available GPU accelerates inference — or run entirely on CPU without hardware requirements.
- Privacy-First Local Execution
- All inference runs on your infrastructure with zero data leaving the machine — no telemetry, no cloud dependencies, and no API keys sent to third-party services for complete data sovereignty.
Features
- Multi-Modal AI Pipeline
- Text generation, image synthesis with Stable Diffusion and Flux, text-to-speech, speech-to-text, vision analysis, video generation, and embeddings through one unified API server.
- Built-In AI Agents
- Autonomous agents with tool use, RAG retrieval, MCP integration, and skill-based workflows execute complex multi-step tasks directly in the web interface or via API.
- Model Gallery
- Curated YAML configurations for hundreds of models install with a single command, automatically downloading weights and configuring optimal backend settings for your hardware.
- P2P Distributed Inference
- Federated inference shards models across multiple machines, enabling execution of models larger than any single node's memory through peer-to-peer coordination without centralized orchestration.
- Web Interface with MCP Apps
- Built-in web UI for chat, image generation, and model management plus interactive MCP tool UIs that expose backend capabilities through visual interfaces.