Xinference lets a team swap a hosted model API for models running on its own machines, with the same client code and a web console for managing them. The Launch Model page lists 300+ models across language, vision, embeddings, reranking, images, audio, video, and experimental world generation, and a Recommend configuration button fills in engine, size, format, and quantization for your hardware. Custom models, including traditional machine learning models, can be registered beside the built-in ones. Chat models can run on vLLM, SGLang, TRT-LLM, Transformers, llama.cpp, or MLX across CPUs, GPUs, and Apple Silicon. Workers and replicas can be added to serve one large model across several machines. Running Models shows each replica and a built-in chat window, and Performance Tests measure throughput and latency at several concurrency levels, with saved runs to compare. Endpoints speak the OpenAI and Anthropic protocols with streaming and tool calling where a model supports it. API keys, user management, and an audit center cover access control. The interface is available in five languages. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Open-weight language models get a private chat service on your own server, and llama.cpp serves its browser UI the moment the server starts. Conversations appear in a sidebar with search, and each answer shows its token count, generation time, and tokens per second, so you can see how a model really performs. Code replies render in highlighted blocks with a copy button, and messages can be edited, regenerated, forked, or deleted. A Models page lists what the server can run with context sizes and load status, and Settings cover themes, system messages, display options, tools, agentic behavior, import and export, and sampling controls for temperature, top K, top P, and penalties. Remote MCP servers connect by URL, and uploads let multimodal models read files and images. Behind the UI sit OpenAI and Anthropic compatible chat endpoints, embeddings, reranking, function calling, JSON schema constrained output, parallel decoding, and continuous batching for several users at once. Quantized models run on plain CPUs, and official images cover CUDA, ROCm, Vulkan, and other accelerators. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.