With over 48,000 GitHub stars and monthly releases since March 2023, LocalAI is the self-hosted AI engine that replaces every OpenAI endpoint with a single Docker container running on your own infrastructure — serving chat completions, image generation, text-to-speech, speech-to-text, embeddings, vision, video generation, and function calling through identical API schemas that require zero application code changes. The composable backend architecture isolates each inference engine as a separate gRPC service running in its own OCI container, so llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, Stable Diffusion, and Flux install on demand without touching the core, can run on separate machines, and a fault in one never affects others. Hardware acceleration spans NVIDIA CUDA 12 and 13, AMD ROCm, Intel oneAPI/SYCL, Apple Silicon Metal, Vulkan, and NVIDIA Jetson L4T — or runs entirely on CPU without any GPU. Built-in AI agents support autonomous tool use, retrieval-augmented generation, Model Context Protocol integration, and skill-based workflows directly in the web interface. The model gallery provides curated YAML configuration files for hundreds of models that install with a single command, while P2P federated inference distributes model shards across multiple machines for running models larger than any single node's memory. Multi-user API key authentication with quotas and role-based access enables team deployments. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

OpenLLM serves any large language model as an OpenAI-compatible API endpoint from a single CLI command, handling model download, backend selection, quantization, and port binding automatically. It supports the full spectrum of popular models including Llama 3.3, Qwen2.5, DeepSeek, Mistral, and Phi3, choosing between vLLM and PyTorch inference backends based on hardware capabilities. When vLLM is available, continuous batching with PagedAttention achieves up to 23x throughput improvement over naive serving, while GPTQ and bitsandbytes quantization reduces memory requirements for GPU-constrained deployments. The server exposes a RESTful API on port 3000 with full OpenAI client library compatibility, enabling drop-in replacement for commercial providers in any application using the standard chat completions format. A built-in web chat UI at the /chat endpoint provides immediate interactive testing without external clients. Custom model repositories allow teams to maintain private catalogs of fine-tuned models alongside the default repository that tracks the latest releases. Deployment workflows generate production-ready Docker images automatically, with Kubernetes manifest support for orchestrated scaling. Native integration with LangChain and LlamaIndex supports RAG pipelines, Transformers Agents enables tool-calling workflows, and HuggingFace Hub handles model discovery. Server-Sent Events enable real-time token streaming across all API endpoints. Backed by BentoML's production ML infrastructure. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Your prompts never have to leave the building: TextGen runs open large language models on your own hardware behind a chat interface and an API that speak fluent OpenAI. Load GGUF or Hugging Face weights through llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, or TensorRT-LLM, and switch models or backends from the interface without restarting the server. Chat mode gives you conversations with characters and system prompts, notebook mode behaves like a familiar completion editor, and both accept image attachments for vision models and documents for context. Models that support tool calling can search the web, fetch pages, and do math through single-file Python tools, and MCP servers plug in for larger tool catalogs. Start it with the API flag and the same model answers at chat, completions, and messages endpoints that mirror OpenAI and Anthropic, complete with streaming, logprobs, embeddings, and optional API key authentication. Portable builds for Linux, Windows, and macOS bundle every dependency, so the distance from download to a running model is an unzip and one shell command. Because everything is local, prompts, documents, and outputs stay on machines you control, and parallel request handling lets a single loaded model serve a whole team's tools at once. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

Imp runs large AI chat models privately on one NVIDIA RTX 50 or RTX PRO graphics card, exposing them through OpenAI and Anthropic compatible APIs plus a built-in browser chat. Start it with a single Docker command, point it at a model, and the chat opens in your browser with saved conversations and a live readout of generation speed. Coding agents such as Claude Code and Codex connect by changing one base URL, so your editor talks to the model on your own hardware instead of a rented cloud endpoint. Tool calls, structured JSON output, streaming responses, and reasoning models all work over the wire, and requests can be protected with an API key, per-IP rate limits, and concurrency caps. Load GGUF files you already have or NVFP4 and FP8 SafeTensors checkpoints straight from Hugging Face with no conversion step before the first answer. When a model is bigger than the card, mixture-of-experts weights stay in system RAM while the busiest experts are cached on the GPU, so a 123.6 GiB model still answers from a 32 GB card. Supported models can also read images alongside text, and tokenize, health, and model listing endpoints make the server easy to wire into scripts and monitors. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Large language models get a polished front end that can run fully offline: Open WebUI is the self-hosted front end of choice. It talks to local model runners, primarily Ollama, and to any OpenAI-compatible API, so LM Studio, vLLM, Groq, Mistral, OpenRouter, and cloud providers all plug into the same chat interface and can be mixed per conversation. RAG is built in: upload files to knowledge bases or reference them in chat with the # command, backed by a choice of nine vector databases (ChromaDB and PGVector officially maintained) and multiple extraction engines including Tika and Docling, with hybrid BM25-plus-vector search and cross-encoder reranking. Web search results from providers like SearXNG, Brave, and Tavily inject directly into conversations. Extensibility comes from Python tools and functions that run inside the chat, a Pipelines plugin framework, and native MCP support. Multi-user features include RBAC, SSO, and group permissions, and the instance itself exposes an OpenAI-compatible API your own apps can call.

Your inference server should not sit mostly idle while a model crawls along at ten tokens per second, so ReadyLLM gives you a visual way to deploy, watch, and tune LLMs until the hardware is actually earning its keep. Add the machine under your desk or a remote box over SSH, let hardware detection read its GPU, VRAM, CPU, and memory, and get model recommendations that fit what you own instead of wishful downloads. A model store catalogs text, image, and video models with parameter counts, precision, resolutions, and real weight filenames, background downloads from multiple sources, and integrity checks when they land. One click starts or stops llama.cpp, vLLM, or SGLang services, installs a missing engine matched to your GPU backend, and leaves runtime parameters open for manual tweaks. Live curves track GPU utilization, memory, throughput, and latency as requests flow. When speed disappoints, two-phase auto tuning searches the knobs that matter, checks VRAM feasibility before it wastes your time, and benchmarks honestly against latency, throughput, or prefill goals; an AI tuning mode can reason through the same loop across several rounds. Beyond chat models, ComfyUI generation covers text-to-image and text-to-video, storyboards, shot-by-shot long video, and upscaling, with prompts optionally expanded first. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Running local models usually means choosing between a polished app that hides the engine and a terminal full of flags you half remember. LlamaForge refuses the trade: it installs and drives the official llama.cpp release itself, then puts a browser panel over all of it, every one of the 200-plus llama-server flags your build supports, grouped, searchable and editable per model. Before you load anything, each model is drawn into your GPUs at its real footprint, so the out-of-memory surprise happens on screen instead of in a crashed server; before you download anything, every quant on Hugging Face gets a fit rating for your VRAM, FITS, TIGHT or CPU OFFLOAD. Auto-Forge then measures placements, KV cache types and speculative drafters on your own hardware and keeps the fastest configuration, checking that a drafter still produces identical tokens before applying it with one click, and undoing it with one click. Keep several models loaded at once and a planner places each on the GPUs that fit it; when a game launches, Game Mode unloads your models and brings them back when you quit. Chat in llama.cpp's own interface, join from your phone by QR code, duel two models blind in the Arena under an Elo board, and let Embers, small local agents, keep a wiki and write scheduled briefs counting only facts they can quote verbatim. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

ODS turns a GPU machine or server you already own into a complete private AI stack with a single install command, replacing a shelf of separate tools with one system that serves models, chat, agents, automation, and search from your own hardware. The installer reads your GPU, VRAM, memory, and disk, then downloads a GGUF model matched to that hardware tier, and a compact bootstrap model starts answering within minutes while the full model finishes downloading. From the web dashboard you watch every service's health, ports, and resource use, restart what misbehaves, and check storage and updates without touching a terminal. You chat with local models in Open WebUI, hand background work to the Hermes and Portal agents, and build automations in n8n that connect the stack to more than 400 other apps. Documents you add become searchable knowledge through Qdrant vector storage, so answers cite your files instead of the model's memory. SearXNG and Perplexica answer web questions without trackers or accounts, Whisper transcribes speech, Kokoro reads text aloud, and ComfyUI generates images, all on the same box. Every bundled service is an extension you can enable, disable, or replace, and a Privacy Shield scrubs personal data before anything reaches an optional cloud model through the LiteLLM gateway. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

One dashboard and one API address for every local model you run, across llama.cpp, vLLM and MLX: that is llamactl. Create an instance in the web UI, pick a backend, and the server assigns ports and API keys, starts the model, and shows its health next to every other instance you run. Models download straight from HuggingFace inside the dashboard, in GGUF for llama.cpp or Safetensors for vLLM, so stocking a new model never means juggling command lines and file paths. Your applications keep talking to familiar endpoints, because llamactl exposes OpenAI chat completions and Anthropic messages routes and sends each request to the model named in the call. A llama.cpp router instance can serve several models from one process and load them on demand, and idle instances stop on a timer or make room through LRU eviction when the GPU gets crowded. Groups cap how many heavy models run at once, backends can live in Docker containers with GPU access, and instances on other machines appear in the same central dashboard with automatic routing. API keys minted in the UI guard both inference and management calls. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One endpoint for every inference box you own: Olla is a proxy and load balancer that fronts your self-hosted LLM backends and routes each request to a healthy node. Your apps keep calling a single OpenAI-compatible address on port 40114, and Olla picks the backend by priority, response rate, and live health. When a node drops, requests retry on the next endpoint automatically and the failed node rejoins after recovery, with circuit breakers keeping bad nodes out of rotation. Multi-turn conversations stick to one backend so its KV cache stays warm across the chat. Model lists from every provider merge into one catalogue, and the same model served by several nodes appears once, ready to route anywhere. Backends with native Anthropic support receive Messages API calls untouched, and other backends get automatic translation, so Claude Code and similar tools can drive your local models. Text embeddings and rerank requests route to the host that serves them, and outbound authentication covers backends that expect API keys. The embedded Herd Dashboard shows fleet health, per-endpoint latency, and the discovered model inventory from the binary itself. The whole proxy runs in under 50 MB of RAM. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

With over 33,000 GitHub stars and a codebase written in 92.9% Rust for maximum performance and memory safety, Tabby is the most widely adopted self-hosted alternative to GitHub Copilot — delivering real-time code completions entirely on your own infrastructure with zero code leaving your network. Deploy a single Docker container on any NVIDIA CUDA, Apple Silicon Metal, AMD ROCm, or CPU-only server and connect VS Code, JetBrains IDEs (IntelliJ, PyCharm, WebStorm, GoLand), Vim, Neovim, and Emacs through native extensions. The completion engine supports a curated registry of models including StarCoder2 (1B to 15B parameters), DeepSeek-Coder, CodeLlama, CodeGemma, Qwen2.5-Coder, and Mistral Code — swappable at runtime through the admin dashboard without redeployment. Repository indexing parses your Git repositories and feeds project-specific types, function signatures, and patterns into completion context via RAG, producing suggestions that understand your codebase rather than generic boilerplate. The Answer Engine provides instant responses to code queries within the IDE, while inline chat enables contextual code editing and explanation without switching windows. The admin dashboard manages per-developer API tokens, usage analytics, and model configuration. Enterprise features include SSO via LDAP, OAuth, and SAML, role-based access control, and audit logging for compliance environments. A single RTX 4090 workstation serves a team of 10-15 developers with sub-500ms completion latency. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Self-hosting a language model is only half the job; people still need somewhere pleasant to talk to it, and Chat UI is that window, the same interface that powers HuggingChat. Point it at any OpenAI-compatible endpoint, your own Ollama or llama.cpp server, the Hugging Face router, or a hosted gateway like OpenRouter, and the models it serves appear in the picker automatically, with no per-model configuration. Conversations are saved to your database and stay browsable in the sidebar, so a thread from last week can be reopened, continued, and compared against a fresh answer from another model. Multimodal models accept image uploads right in the composer, and models that support it can call MCP tools mid-conversation to fetch data or run actions. For a shared team deployment, OpenID Connect sign-in keeps individual histories and settings separate, and per-user defaults travel with each account. Everything a consumer chatbot would keep on someone else's servers, prompts, uploaded files, and usage history, stays in the MongoDB you control on the same machine. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Every major model provider behind one ChatGPT-style interface: LibreChat spans OpenAI, Anthropic, Google, Azure, AWS Bedrock, Vertex AI, Groq, Mistral, OpenRouter, DeepSeek, and any OpenAI-compatible endpoint including local Ollama. You can switch models mid-conversation and compare providers without changing tools. Its Agents framework builds no-code custom assistants with tool access via Model Context Protocol servers, file search over uploaded documents through an optional pgvector-backed RAG service, and a sandboxed Code Interpreter that executes Python, JavaScript, Go, C++, Java, PHP, and Rust. Artifacts render React components, HTML, and Mermaid diagrams directly in chat, and image generation works through DALL-E and other configured providers. Multi-user support is enterprise-grade, with OAuth, SAML, LDAP, and two-factor authentication, per-user conversation history in MongoDB, and Meilisearch-powered search across all messages and files, plus reusable presets, forkable threads, and persistent memory across conversations. The economics favor teams: instead of a ChatGPT Plus seat per person, everyone shares one instance billed per API token, with access to every provider rather than one - and providers see individual API calls, not your accumulated organizational knowledge. Deployment is Docker Compose; API keys and endpoints are configured through .env and librechat.yaml.