FreeLLMAPI collapses the chaos of 29 free LLM providers — Google AI, Cerebras, Groq, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare Workers AI, NVIDIA NIM, HuggingFace, SiliconFlow, Reka, Z.ai, and more — into a single /v1 endpoint that speaks both OpenAI and Anthropic protocols. The smart router selects the best available model for each request, automatically fails over to the next provider when rate limits hit, and tracks per-key token consumption so you never exceed a free-tier cap. Keys are stored with AES-256-GCM encryption and clients authenticate using a single unified bearer token, never exposing upstream provider credentials to downstream applications. The catalog tracks 251 model families across 358 provider/model endpoints with approximately 4 billion tokens per month of aggregate free-tier capacity, auto-refreshing from a signed manifest at freellmapi.co twice daily without requiring git pulls. Beyond chat completions, the proxy handles embedding, image generation, and audio/TTS endpoints, plus structured outputs with JSON schema forwarding, JSON healing, and format-ignore failover. An integrated MCP server at /mcp provides gateway introspection for coding agents, while the self-hosted OpenAPI reference at /v1/docs documents every route. Compatible with OpenAI SDKs, LangChain, LlamaIndex, Continue, Claude Code, and Hermes — just change base_url. Deploy via Docker, npm, or build from source. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Routerly is the one front door for every AI model your apps use: point your OpenAI or Anthropic client at it, and it chooses the right model for each request, enforces your budget, and keeps the receipts. Nothing in your code changes except the base URL, and from that moment streaming chat completions and messages flow to OpenAI, Anthropic, Gemini, Mistral, Cohere, xAI or your own Ollama models behind a single token. Nine routing policies score every candidate in parallel: cheapest, fastest, healthiest, most capable, biggest context, least rate-limited, fairest, most budget left, and one that asks a small model to judge which model fits the task. When the winner errors or times out, the next candidate takes over automatically, and the recorded routing trace shows exactly why each model won or lost. Guardrails can block or log content that breaks your rules, and phone numbers, card numbers, SSNs, IBANs and emails can be scrubbed out before anything reaches a provider. Every call is logged with tokens, USD cost and latency, and hard limits per model, project and token reject overspending requests before a provider can charge for them. Projects keep separate tokens, model lists and budgets for each team or tool, role-based permissions decide who may change what, and alerts land in Slack, email, webhooks or PagerDuty when budgets or providers misbehave. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

One OpenAI-compatible endpoint, 1,600+ models behind it: Portkey Gateway sits between your app and every major LLM provider, deciding in real time where each request should go. When a provider times out or returns errors, automatic retries with exponential backoff kick in, and fallback rules quietly move traffic to your second-choice model before users notice anything. Load balancing spreads calls across multiple API keys and providers by weight, conditional routing sends each request down a different path based on its content or metadata, and per-request timeouts cut off providers that hang. Guardrails inspect inputs and outputs as traffic passes through: block banned words, screen for sensitive content, and enforce the checks your compliance team asked for, with dozens of pre-built rules ready to attach. Text is only the start: the same API shape reaches vision models, speech-to-text, text-to-speech, image generation, and realtime websocket APIs, so multimodal features stop multiplying your integration work. Routing policies live in JSON configs you attach per request or per client, which means experiments, cost controls, and provider migrations ship as configuration changes, not redeploys. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

New API is a Go-powered LLM gateway that funnels over 40 AI providers, from OpenAI and Anthropic to Google Gemini, Azure, AWS Bedrock, DeepSeek, and Mistral, through a single OpenAI-compatible endpoint with intelligent routing and enterprise cost controls. Bidirectional format conversion translates between OpenAI Chat Completions, Claude Messages, and Gemini GenerateContent APIs transparently, so clients call any provider using their preferred format while the gateway handles the rest. Routing logic supports weighted random channel selection, priority-based failover, and automatic retry on provider errors to maximize uptime. The React admin dashboard shows usage charts, token consumption tracking, per-model cost breakdowns, and real-time request monitoring. Multi-tenant architecture includes three role levels, token-based authentication, per-user quota management, and a three-phase billing system with tiered pricing via a custom expression language. Multimodal support covers text completion, vision, embeddings, text-to-speech, speech-to-text, image generation, and creative task providers like Midjourney-Proxy and Suno-API. The interface ships in five languages including English, Chinese, Japanese, and French. Docker deployment runs with SQLite or MySQL for persistence. 45,000+ GitHub stars. AGPL-3.0 licensed.

One OpenAI-compatible API fronts more than 250 models across 40 providers in LLM Gateway, with routing, cost analytics, and key management in a single dashboard. Existing OpenAI SDK code keeps working: change the base URL and model name to reach OpenAI, Anthropic, Google, xAI, DeepSeek, Groq, and dozens more. Every request lands in an activity log with its model, cache status, token counts, duration, and cost, and opening a request shows time to first token, throughput, and cached token usage. Spend splits by model, provider, project, and API key over 7 or 30 days, so the expensive endpoint stops hiding inside one monthly invoice. Scoped API keys carry spend caps and recurring limits, and requests stop automatically at the cap. Provider errors, timeouts, and rate limits trigger a retry on another provider for the same model before the caller hears about it. Coding agents and automation tools get their own usage views with requests, tokens, and spend per tool. Self-hosting runs the unified Docker image or a Kubernetes Helm chart on your own network. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

An upstream dies mid-stream. The session does not have to. ccLoad is a self-hosted AI gateway that stands between your coding agents and every key, relay, and subscription you own, retrying the next source before the first byte reaches your client. Claude Code, Codex, Gemini, and OpenAI-compatible tools all point at one address and present one ccLoad token. Behind it, each source becomes a channel with a model list, a priority, and health-based ordering. Requests are matched by the model name the client asked for. Protocol differences disappear in transit, with Anthropic, OpenAI, Codex Responses, and Gemini converted in either direction, streamed responses included. Failure gets a narrow blast radius. A revoked key cools down alone, a failing model cools down alone, and a dead URL cools down alone, each with exponential backoff that respects upstream reset times. Errors disguised as successful responses still trigger failover. Subscription accounts work as channels beside API keys, tokens refresh on their own, and quota use is tracked against each provider's reset window. Every request lands in the logs with its cost, priced with cache reads, cache writes, and long-context tiers included. Tokens carry model restrictions, budgets, and concurrency caps. Channels carry daily spend limits. Sensitive text can be masked on the way out and restored on the way back. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One endpoint, every model you run, and a government-grade paper trail for each request. OpenGateLLM sits in front of your AI infrastructure as the single door: applications call one OpenAI-compatible API, and the gateway decides which backend answers, a self-hosted vLLM cluster, an Ollama box in the corner, a Hugging Face TEI embeddings service, or a hosted provider when you allow one. You publish friendly model names as routers and attach real provider APIs behind them, then load-balance across those backends by shuffle or least-busy so a second GPU node starts earning its keep the moment you register it. Chat completions, embeddings, reranking, OCR, and audio transcription each get their standard endpoint, so existing OpenAI SDK code, LangChain, and LlamaIndex projects point at a new base URL and keep working. Governance is built in rather than bolted on: organizations, users, and roles control who may call which model, API keys authenticate every request, and rate limiting keeps one enthusiastic team from eating the whole cluster. Usage monitoring follows every inference, and carbon footprint estimates attach to models through their hosting zone and parameter counts. Retrieval-augmented generation against Elasticsearch or Qdrant plugs into the same surface. No chat history is stored, which is rather the point when the data is yours. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Instead of scattering provider credentials and vendor-specific SDK logic across dozens of microservices, engineering teams route every LLM call through GoModel, an open-source AI gateway that unifies 34 model providers behind OpenAI-compatible and Anthropic-compatible endpoints. Pointing existing OpenAI or Anthropic SDKs at the gateway unlocks chat completions, the Responses API, Anthropic Messages, embeddings, batch jobs, and realtime audio across OpenAI, Anthropic, Gemini, Bedrock, Azure, Groq, DeepSeek, Ollama, and vLLM. Platform operators publish virtual model aliases that distribute traffic via weighted round-robin or lowest-cost routing while rotating multiple API keys per provider and failing over through circuit breakers during upstream outages. Sticky session routing keeps multi-turn agent conversations on the same target to preserve warm prompt caches, while exact-match and vector semantic caching via Qdrant, pgvector, Pinecone, or Weaviate cut duplicate token costs. Versioned workflows and pre-dispatch guardrails enforce prompt injection rules, rate limits, and hard USD spend budgets across hierarchical tenant user paths before requests are dispatched. The built-in admin dashboard offers live audit logs, per-tenant cost analytics, LiteLLM key migration, an interactive request playground, Model Context Protocol tool aggregation, and OpenTelemetry and Prometheus metrics. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One endpoint for every model account you own: 9Router Pro sits between your applications and a fleet of LLM providers, decides which account answers each request, and keeps the whole operation visible on a live dashboard. Its signature trick is prompt injection scoped three ways, globally, per API key, or per model, delivered either as a system prompt or as a scripted chat history that primes the model far harder than instructions alone. Keys are products here: each carries a rolling token budget over 24 hours, 7 days, or 30 days plus rate limits per minute, hour, and day, so a resold plan enforces itself with polite 429 responses. Routing stays healthy on its own, with accounts tiered as clean, suspect, or cooldown, least-recently-used spreading across the fleet, flaky upstream responses retried against the next account, and proxy pools health-checked with one-click cleanup of the dead. The monitoring dashboard leads with VPS specs, fastest and most-used providers, top API key, and usage spectrum charts, while backup and restore cover providers, combos, keys, and settings section by section. Everything runs as a single application with one SQLite file and a dashboard localized into 34 languages. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Backed by 56,000+ GitHub stars and over 240 million Docker pulls, LiteLLM delivers the open-source AI gateway trusted by Netflix, Lemonade, Rocket Money, and thousands of engineering teams to route every LLM request through one unified API. The Rust-core gateway adds sub-millisecond overhead per request with 8ms P95 latency at 1,000 RPS, 15x throughput improvement and 11x lower memory footprint compared to Python-only proxies. A single OpenAI-compatible endpoint connects to 100+ providers and 1,800+ models spanning OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Vertex AI, Hugging Face, vLLM, Nvidia NIM, Ollama, and Mistral with day-zero support for new model releases. The Auto Router V2 classifies request complexity across four tiers using rule-based scoring, semantic keyword matching, and adaptive Thompson sampling to route each request to the most cost-effective model without API calls or training data. Virtual API keys enable multi-tenant governance with per-team, per-user, and per-project cost tracking, budget caps with automatic fallback rerouting, and role-based access control. Built-in guardrails provide PII masking, prompt injection detection, and model-graded evaluation before requests reach providers. The Agent Gateway extends routing from model calls to agent workflows with MCP server integration. Observability integrates with Langfuse, Arize Phoenix, OpenTelemetry, and MLflow for complete request tracing. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Respan Gateway is a single-binary LLM gateway that lets the SDK you already use call any AI model from any provider, translating between OpenAI, Anthropic, and Gemini API formats on the way through. Point an OpenAI, Anthropic, or Gemini client at it and you reach chat models on sixteen providers, plus any OpenAI-compatible endpoint of your own, with streaming, tools, images, and reasoning translated. Traffic control works the way operators think: name a route, send it to the cheapest or fastest provider, retry on failure, fall back to a second vendor, or hedge a slow call by racing another provider. Models that never had tools get them anyway, because MCP servers and built-in search, browsing, code, document, and memory tools run inside ordinary chat requests behind the gateway. Realtime voice crosses vendors too, with an OpenAI Realtime client able to talk to Gemini Live, alongside images, speech, transcription, embeddings, and video through one endpoint. A live request log in the web UI shows the client key, model, provider, status, tokens, cost, and latency of every call, so runaway agents and surprise bills are visible immediately. Everything is a plugin on one public Rust API, from routing and pricing to tracing, with examples for guardrails, PII masking, and billing. One TOML file configures keys, providers, routes, and tools, validates before it starts, and reloads on signal. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Bifrost is an open-source AI gateway that unifies 23+ LLM providers into a single OpenAI-compatible endpoint with automatic failover, semantic caching, and built-in cost governance, so one provider going down never takes your production AI application with it. Point your existing OpenAI or Anthropic SDK at Bifrost's local endpoint and gain access to OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Groq, Mistral, and Ollama without changing application code. Define fallback chains that automatically switch providers when one returns errors or exceeds latency thresholds, keeping response times stable during outages. The built-in web dashboard at port 8080 lets you configure providers, create virtual API keys, monitor live request traffic, and review analytics without editing configuration files. Semantic caching combines exact hash matching with vector similarity search via Weaviate, serving cached responses for identical or paraphrased prompts in sub-millisecond time to cut costs on repetitive workloads. The MCP gateway connects AI agents to external tools like filesystems, databases, and web APIs, exposing them to clients such as Claude Desktop and Cursor with per-key allow-lists. Four-tier budget hierarchy at customer, team, virtual key, and provider levels enforces spend caps, rate limits, and model restrictions across your organization. Extend functionality through custom Go plugins for analytics, monitoring, or security middleware. Native Prometheus metrics and OpenTelemetry distributed tracing give operations teams full production observability. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Your apps keep using the OpenAI SDK while Antirion, a self-hosted AI gateway, decides which provider answers, what each team may spend, and what gets logged. Clients speak the OpenAI, Anthropic, Gemini, or Bedrock APIs, and Antirion translates between them, so a Claude deployment can answer an OpenAI-style call, streams included. Add providers such as Amazon Bedrock, Vertex AI, Mistral, Groq, DeepSeek, or a vLLM server, then publish them as named Models. Routing rules send traffic by team or project with fallback chains, weighted splits, least-cost or least-latency targets, retries, hedging, and circuit breakers. Budgets and rate limits in dollars or tokens apply per organization, team, project, and key, with projections and alerts before money runs out. Guardrails check for personal data, secrets, keywords, and regex matches, then block, redact, flag, or divert. Exact and semantic caching answers repeated prompts without calling a provider. The request log records every attempt, route, token count, cost, and guardrail verdict, with Prometheus metrics and OTLP traces. An MCP gateway and A2A proxy put tools and agents behind the same keys. Sign in with OIDC, SAML, or GitHub, provision users with SCIM, and review every change in the audit log. Changes you choose can require a change request or a second person's approval, and the Playground sends requests from the console. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Juggling separate API keys, incompatible payload schemas, and unpredictable rate limits across OpenAI, Anthropic, and Google Gemini slows down every AI workflow, and Octopus solves it by consolidating all your upstream model providers into one self-hosted API gateway. Point Claude Code, OpenAI Codex CLI, or any SDK client at a single gateway URL and translate requests transparently between OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Gemini endpoints without modifying client code. Group multiple provider channels under a unified model alias so your coding agents automatically fail over to backup providers and shield active tasks from upstream HTTP errors or rate limits. Watch live end-to-end request routing on the interactive topology canvas as prompts flow from client applications through model groups to specific upstream channels. Synchronize available model catalogs directly from connected providers and pull up-to-date token pricing from models.dev, or define custom per-million input, output, and cache-read rates for private deployments. Inspect detailed request logs, monitor token consumption trends, and audit daily API spend across every channel from the web dashboard. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

Codex LB Enhanced is a session-aware gateway that pools your ChatGPT accounts so a long Codex session survives quota limits, dropped connections, and paused accounts without losing the conversation. It serves Codex desktop, CLI, and IDE clients through the native Responses protocol and WebSocket transport, and answers any OpenAI-compatible client through the familiar /v1 endpoints. When an account runs out of quota, gets suspended, times out, or drops mid-stream, the gateway classifies the failure, removes that account from the current request, and selects the next one through your load-balancing policy. Verified request history, including interrupted tool-call context, is replayed across the account switch when it is safe, so the client sees a slower answer instead of a dead stream. Every API key carries its own policy: time windows from five hours to monthly, limits by credits, tokens, or cost, and model-specific rules. Usage appears in native Codex quota headers and in a /v1/usage view backed by the same data, so editors and automation agree on what is left. Diagnostics correlate each request with its upstream account, transport, failover stage, and retry count, and a web dashboard manages accounts, keys, usage windows, request logs, and recovery reports. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Anyone who pays for Claude Code, Codex, and Gemini CLI separately ends up juggling three endpoints, three sets of keys, and three kinds of rate-limit surprise, and CliRelay collapses all of that into one self-hosted gateway with a control panel. Clients speak the OpenAI Chat or Responses, Anthropic Messages, or Gemini protocol, and CliRelay translates each request for whichever upstream account is healthy. Channel groups limit where each key may send traffic, round-robin, fill-first, and session-sticky scheduling spread it, and cooldowns with automatic failover move requests off accounts that have run out of quota. Add upstreams from the /manage console through browser sign-in, device codes, or bulk credential import, covering Claude, Codex, Gemini, Antigravity, Vertex AI, AWS Bedrock, Grok, Qwen, Kimi, and any OpenAI-compatible service. Portal accounts own client keys with daily, period, and lifetime quotas, RPM and TPM limits, and per-model pricing. A monitor center scores health and charts live traffic, P50 to P99 latency, and time to first token, while every request lands in a searchable log with tokens, latency, and cost. Tenants, roles with resource.action permissions, content moderation profiles, an outbound proxy pool, and an audit trail make shared deployments workable, and the console speaks English, Simplified Chinese, and Russian. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Oh My CPA gives a CLIProxyAPI gateway a proper control room, where providers, OAuth credentials, client keys, model routing, and the cost of every request are managed and watched from one self-hosted console. You add providers such as Codex, Claude, Gemini, xAI, DeepSeek, Vertex AI, and OpenAI-compatible services, each with credentials, retry and error policies, priority, weight, and an enable switch the gateway actually enforces. OAuth accounts are signed in from the console, with quota window readings, refresh schedules, and per-credential model aliases kept next to the keys they belong to. Client keys can be created, named, and revoked, and those names follow the traffic into request records and filters so you always know which key spent what. The dashboard follows request volume, token throughput, cache hit rate, and cost from the last fifteen minutes out to ninety days, with a year-long token heatmap for the long view. Insights compare keys, models, providers, and credentials against the previous period, while the live routing view shows which credential each request travelled to and whether it was rerouted. Every request stays searchable, with duration, time to first token, token breakdown, and the raw log in a detail drawer. A playground tests any routed model with text and images before your users do. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Headroom answers the question every multi-plan AI subscriber asks mid-task: which of my accounts still has room to work? It puts the live limits, reset times, credits, and spend of Claude, Codex, Cursor, Copilot, Grok, Antigravity, and Vercel AI Gateway accounts on one self-hosted dashboard instead of seven provider pages. Connect each account once, through its official CLI sign-in or a credential import, and a background scheduler refreshes every figure on schedule, renews tokens before they expire, and keeps 90 days of readings. Five-hour sessions, weekly pools, per-model limits, credit balances, banked resets, and on-demand spend sit side by side, and unreported figures show as unknown rather than a fake zero. Compare ranks same-provider accounts by the room they have left and names the one to use next, while Timeline lays every reset window on a two-week calendar. Analytics charts when you work, when you run dry, and whether each plan fits, and Wallet totals subscriptions, usage spend, and top-ups in one currency with renewals and budget alerts. In-app notices, Telegram messages, and signed webhooks warn about low limits, expiring resets, and broken sign-ins, and optional automations can spend a banked Codex reset or log a credit top-up the moment it lands. Personal, work, and side-project accounts stay separate under one owner login with passkeys and a Privacy Mode that blurs emails on screen. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

OpenLLM serves any large language model as an OpenAI-compatible API endpoint from a single CLI command, handling model download, backend selection, quantization, and port binding automatically. It supports the full spectrum of popular models including Llama 3.3, Qwen2.5, DeepSeek, Mistral, and Phi3, choosing between vLLM and PyTorch inference backends based on hardware capabilities. When vLLM is available, continuous batching with PagedAttention achieves up to 23x throughput improvement over naive serving, while GPTQ and bitsandbytes quantization reduces memory requirements for GPU-constrained deployments. The server exposes a RESTful API on port 3000 with full OpenAI client library compatibility, enabling drop-in replacement for commercial providers in any application using the standard chat completions format. A built-in web chat UI at the /chat endpoint provides immediate interactive testing without external clients. Custom model repositories allow teams to maintain private catalogs of fine-tuned models alongside the default repository that tracks the latest releases. Deployment workflows generate production-ready Docker images automatically, with Kubernetes manifest support for orchestrated scaling. Native integration with LangChain and LlamaIndex supports RAG pipelines, Transformers Agents enables tool-calling workflows, and HuggingFace Hub handles model discovery. Server-Sent Events enable real-time token streaming across all API endpoints. Backed by BentoML's production ML infrastructure. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Your AI application is burning through API tokens faster than you can refresh the billing page, and you have no idea which prompt template is responsible. OpenLIT plugs that visibility gap with a self-hosted observability platform built specifically for LLM workloads. Add one line of code to instrument 90+ LLM providers, agent frameworks, and vector databases, then watch every request flow through a tracing dashboard that shows tokens consumed, latency measured, and dollars spent per call, per model, per environment. The requests view lists every LLM interaction with provider, model, cost, and token breakdown in a filterable table, while the trace detail panel lets you drill into individual spans to read the exact prompt sent and response received. Prompt Hub turns prompts into versioned artifacts you deploy, rollback, and A/B test without touching application code. OpenGround compares models side by side on the same input, so you can evaluate cost-versus-quality tradeoffs before committing to a provider. Automated evaluations run LLM-as-a-judge scoring on live production traces, flagging hallucinations, bias, and toxicity in real time. The Vault stores and rotates API keys centrally so secrets stay out of your codebase. Custom dashboards let you build drag-and-drop monitoring views with charts, stat cards, and tables backed by SQL queries against ClickHouse. GPU utilization, memory, temperature, and power metrics feed into the same platform for end-to-end infrastructure visibility. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.