Categories
Developer Tools Self-Hosted AI AI Gateway API Gateway Cloud Native LLM Proxy Model Routing MCP HostingStars
Forks
Watchers
Developer links
Higress
Send every AI request through one gateway that holds your model providers, API keys, and access rules. The web console starts with a first-run admin setup, then an LLM Provider page where OpenAI-compatible endpoints, Anthropic Claude, Azure OpenAI, Tongyi Qianwen, Moonshot, Zhipu, and other services are added with their tokens. AI Route Config maps path prefixes and model match rules to weighted target services, with token failover and first-packet timeouts to keep traffic moving when a key fails. Consumer Management issues key, OAuth2, and JWT credentials, and each route can limit access to named consumers. Plugin Management lists ready-made AI plugins for caching, context limits, data masking, prompt templates, prompt decorators, load balancing, and intent checks, plus traffic and security plugins that can be configured per route. MCP Management hosts MCP servers behind the same authentication and rate limits so agents call tools through one governed address. Service Sources import backends from Nacos, Zookeeper, Consul, Eureka, static addresses, or domains, and domain and certificate pages handle HTTPS, including free automatic certificates from Let's Encrypt. An LLM proxy for a product team, a shared model endpoint for several agents, or a replacement for an ingress controller all fit this console. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.
Benefits
- One Endpoint For Every Model
- Applications call a single OpenAI-compatible address and the gateway picks the provider, so switching or adding models never requires changing client code.
- Keys Stay Out Of Apps
- Provider tokens live inside the gateway and callers use their own consumer credentials, which keeps secrets out of repositories and makes revoking access simple.
- Failover Keeps Requests Flowing
- Weighted targets, token failover, and first-packet timeouts route around a failing key or provider instead of returning errors to your users.
- Governed Tools For Agents
- Host MCP servers behind the same authentication, rate limits, and audit trail as your model traffic, so agent tool calls stay visible and controlled.
Features
- LLM Provider Management
- Register OpenAI-compatible, Anthropic, Azure OpenAI, and regional model services with tokens, custom base URLs, timeouts, and optional proxy servers.
- AI Route Config
- Match requests by path, header, query, or model name and send them to weighted target services, with per-route consumer access control.
- Consumer Credentials
- Create consumers with key authentication, OAuth2, or JWT credentials and choose which routes each one may call.
- AI Plugin Library
- Turn on caching, context limits, data masking, prompt templates, load balancing, and JSON formatting for any route from one plugin catalog.
- MCP Server Hosting
- Publish MCP servers, including ones converted from OpenAPI specifications, with shared authentication, rate limiting, and observability.
- Service Source Discovery
- Import backends from Nacos, Zookeeper, Consul, Eureka, static addresses, or DNS domains and route traffic to them by name.