Deploy Now

Stars

4,123

Forks

520

Watchers

23

Developer links

Mesh LLM

Spare GPUs and memory scattered across many machines become one OpenAI-compatible API for running open models on hardware you already own. Mesh LLM merges a home lab, an office, or a group of friends into a single pool, so existing chat apps and coding agents work unchanged. Start a node, add more later, and the mesh chooses between running a model locally, routing to a peer, or splitting it across several machines when it is too large for one box. Requests are routed by the model name, a virtual model called mesh picks a compatible one for you, and traffic between nodes travels over encrypted QUIC. Private meshes join by invite token and public ones can be discovered, with an API-only client mode for machines that host nothing. A web console shows the node map, model catalog, connected peers, chat with streaming replies, request logs, and a configuration editor. Goose, OpenCode, Claude Code, and Pi launch with one command and use the mesh as their provider, and plugins add extra backends. Releases cover Linux, macOS, and Windows with CPU, CUDA, ROCm, Vulkan, and Metal runtimes, and a headless flag keeps servers tidy. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Mesh LLM
Mesh LLM
Mesh LLM
Mesh LLM
Mesh LLM
Mesh LLM

Benefits

  • One Endpoint For Every Model
  • Every node exposes the same OpenAI-compatible API, and requests go to the peer that can serve the model named in the call.
  • Run Models Too Big
  • When a model does not fit on one machine, layer splits spread it across several nodes, with the pieces fetched only where each stage needs them.
  • Private By Invite Only
  • Private meshes admit only machines holding your invite token, and encrypted QUIC protects inference requests, replies, and split-model traffic between nodes.
  • Bring Your Own Agents
  • Goose, OpenCode, Claude Code, and Pi each start with one command and use the mesh as their model provider instead of a paid API.

Features

  • Network Map Console
  • Watch nodes, links, capacity, loaded models, and connected peers on a live map, with the API address and join command shown beside it.
  • Built-In Chat
  • Chat with any model on the mesh from the browser, pick one by name or let the mesh choose, and see tokens per second.
  • Request Logs
  • Review request activity and system events on a timeline with filters, completed and failed counts, retained records, and export of the current view.
  • Configuration Editor
  • Inspect runtime, model, and network settings in a categorized editor backed by one config file, with a TOML output tab to preview the result.
  • Automatic Model Selection
  • Send the model name mesh and the system picks a capable model from what peers offer, with optional committee answers through a plugin.
  • Client And Headless Modes
  • Run an API-only client with no local model, or hide the web interface on servers and keep the management API available.

Apps Similar to Mesh LLM