Categories
Self-Hosted Generative AI AI Chat Roleplay Creative Writing Local LLM Runtime AI Inference Server GGUF ModelsStars
Forks
Watchers
Developer links
KoboldCpp
One file runs your own language model and serves it to any browser tab. KoboldCpp loads GGUF models on CPU or GPU. A web UI opens on port 5001. OpenAI, Ollama, Anthropic, and KoboldAI style endpoints answer every app that expects a cloud API. The bundled KoboldAI Lite interface covers chat, instruct, story, and adventure modes in Classic, Messenger, Aesthetic, and Corpo themes, with memory, world info, author's notes, character cards, scenarios, and saveable sessions. A llama.cpp style chat interface and interactive API documentation share the same port. Add a vision projector and the model reads images. Load Stable Diffusion, Flux, or video models for pictures and clips, Whisper for transcription, and Kokoro or Qwen3TTS for spoken replies. Tool calling, web search, retrieval over your own documents, an MCP bridge, and a built-in agent for editing code come included. Context size, GPU layers, batch size, and samplers including DRY and XTC are set from the launcher or the command line. A no-model option points it at a remote provider instead. An API key protects the endpoint before it faces a network. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.
Benefits
- Private Models On Your Hardware
- Prompts, stories, and documents stay on your own server, so sensitive writing and company data never pass through a third party's cloud API.
- One Server For Every App
- OpenAI, Ollama, Anthropic, and KoboldAI compatible endpoints share port 5001, so existing editors, bots, and automation tools connect without custom integration work.
- Tuned To Your Hardware
- Choose context size, GPU layer offloading, batch size, and quantized GGUF files so a modest CPU server or a large GPU machine both run models well.
- Writing And Roleplay Built In
- Memory, world info, author's notes, character cards, and scenario presets turn the web UI into a long-form writing studio, not just a chat box.
Features
- Four Web Interface Themes
- Switch between Classic, Messenger, Aesthetic, and Corpo themes in KoboldAI Lite, or open the llama.cpp style interface on the same port.
- Compatible Local APIs
- Serve OpenAI, Ollama, Anthropic, and KoboldAI style endpoints, with interactive API documentation at the api path for quick testing.
- Image, Voice, And Vision
- Add Stable Diffusion or Flux image models, Whisper speech recognition, text-to-speech voices, and vision projectors for image understanding.
- Tools And Agents
- Use tool calling, web search, document retrieval through TextDB, MCP servers, and an integrated agent that edits code and runs programs.
- Deep Sampler Controls
- Tune temperature, top-p, top-k, repetition penalty, DRY, XTC, and sampler order from the settings panel to shape every generation.
- Scenarios And Save Files
- Start from ready-made stories, adventures, and chats, import character cards, and save or reload sessions as portable files.