Deploy Now

SaaS Alternative

Portkey LiteLLM OpenRouter

Categories

Self-Hosted Developer Tools DevOps AI API Gateway Proxy Load Balancer

Stars

6,560

Forks

706

Watchers

19

Developer links

Last Updated: 09-08-2026
Share on X

GPT Load

GPT Load has become the go-to self-hosted AI gateway for teams managing multiple LLM provider credentials behind a single URL. The Go-built proxy transparently preserves native API formats for OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Gemini — applications swap one base URL and keep their existing SDK integration untouched. Twenty built-in channels cover official APIs (OpenAI, Anthropic, Gemini, xAI), cloud platforms (Azure OpenAI, AWS Bedrock, Google Vertex AI), model services (DeepSeek, Moonshot AI, SiliconFlow, Zhipu AI, Alibaba Cloud, Volcengine, OpenRouter, Groq), and subscription accounts (Codex, Claude, Antigravity, Grok) using the same credential management, scheduling, and health system. The intelligent key pool rotates among valid credentials using atomic counters for fair high-concurrency distribution, automatically blacklists failing keys after configurable thresholds, and recovers them via scheduled health checks — all transparent to clients. Weighted load balancing distributes traffic across multiple upstream endpoints while per-credential proxies route individual keys through different egress paths. The Vue 3 management dashboard provides real-time statistics, group configuration, key testing, request logs with full debugging context, per-model cost estimates, and route inspection. Docker deployment exposes the management interface and proxy on port 3001 with SQLite by default, MySQL or PostgreSQL optional, and Redis for distributed leader-follower cluster scaling. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

GPT Load
GPT Load
GPT Load
GPT Load
GPT Load

Benefits

  • One Gateway for 20+ Providers
  • Unifies OpenAI, Anthropic, Gemini, Azure, AWS Bedrock, Vertex AI, DeepSeek, and 13 more services behind a single proxy URL — applications change one base URL to access all providers.
  • Intelligent Key Pool Rotation
  • Atomic-counter round-robin across valid credentials with automatic blacklisting after configurable failure thresholds and scheduled health-check recovery — fully transparent to clients.
  • Native API Format Preservation
  • Transparently proxies OpenAI Chat Completions, Anthropic Messages, Gemini native, and other formats without transformation — existing SDK code works by swapping one URL.
  • Enterprise Cluster Architecture
  • Distributed leader-follower deployment with MySQL or PostgreSQL and Redis enables horizontal scaling, high availability, and shared credential pools across multiple proxy nodes.

Features

  • Vue 3 Management Dashboard
  • Real-time statistics, group configuration, key testing, request logs with debugging context, per-model cost estimates, and route inspection in a modern web interface.
  • Weighted Load Balancing
  • Distributes traffic across multiple upstream endpoints using configurable weights, with per-credential proxy support for routing individual keys through different egress paths.
  • Smart Failure Handling
  • Automatic retry with next available credential on failure, configurable blacklist thresholds, and background recovery tasks that restore keys after successful health checks.
  • Subscription Account Support
  • Manages Codex, Claude, Antigravity, and Grok subscription accounts alongside API keys using the same credential pool, scheduling, and health monitoring system.
  • Hot-Reload Configuration
  • System settings and group configurations update instantly without restarting the proxy, with a three-tier priority system: group overrides, system settings, and environment defaults.
  • Docker One-Command Deploy
  • Single Docker run command with SQLite embedded by default, optional MySQL/PostgreSQL and Redis, or Docker Compose with full database stack included.