llamactl
One dashboard and one API address for every local model you run, across llama.cpp, vLLM and MLX: that is llamactl. Create an instance in the web UI, pick a backend, and the server assigns ports and API keys, starts the model, and shows its health next to every other instance you run. Models download straight from HuggingFace inside the dashboard, in GGUF for llama.cpp or Safetensors for vLLM, so stocking a new model never means juggling command lines and file paths. Your applications keep talking to familiar endpoints, because llamactl exposes OpenAI chat completions and Anthropic messages routes and sends each request to the model named in the call. A llama.cpp router instance can serve several models from one process and load them on demand, and idle instances stop on a timer or make room through LRU eviction when the GPU gets crowded. Groups cap how many heavy models run at once, backends can live in Docker containers with GPU access, and instances on other machines appear in the same central dashboard with automatic routing. API keys minted in the UI guard both inference and management calls. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.
Deploy