CPA Manager Plus is a self-hosted observability dashboard and management panel that tracks every AI request flowing through your CLI Proxy API gateway, breaking down failures, costs, and account health across providers like OpenAI, Anthropic, xAI, and Codex in one interface. When a request fails, drill into the persistent history to see status codes, affected models, latency, and redacted failure evidence without exposing raw response bodies. The cost analytics engine breaks down token consumption and estimated spend by model, provider, account, API key, project, channel, and time range while tracking input, output, reasoning, cache, and service-tier pricing semantics separately. Model prices sync automatically from models.dev with LiteLLM and OpenRouter fallbacks, and you can add local overrides for aliases or internal models. For teams running Codex or xAI accounts, the health inspector reads quota windows, reset evidence, credential state, and workspace status on a configurable schedule, routing credential failures into an action queue for review rather than letting them silently degrade throughput. Deploy the Lightweight Panel to replace your existing CPA management UI without adding another service, or run Full Mode as a single Docker container that adds the Manager Server with persistent SQLite storage for request history, historical analytics, and automated account inspections. Export or import request history as JSONL for external analysis, and back up the SQLite files alongside your encrypted management keys. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Backed by 56,000+ GitHub stars and over 240 million Docker pulls, LiteLLM delivers the open-source AI gateway trusted by Netflix, Lemonade, Rocket Money, and thousands of engineering teams to route every LLM request through one unified API. The Rust-core gateway adds sub-millisecond overhead per request with 8ms P95 latency at 1,000 RPS, 15x throughput improvement and 11x lower memory footprint compared to Python-only proxies. A single OpenAI-compatible endpoint connects to 100+ providers and 1,800+ models spanning OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Vertex AI, Hugging Face, vLLM, Nvidia NIM, Ollama, and Mistral with day-zero support for new model releases. The Auto Router V2 classifies request complexity across four tiers using rule-based scoring, semantic keyword matching, and adaptive Thompson sampling to route each request to the most cost-effective model without API calls or training data. Virtual API keys enable multi-tenant governance with per-team, per-user, and per-project cost tracking, budget caps with automatic fallback rerouting, and role-based access control. Built-in guardrails provide PII masking, prompt injection detection, and model-graded evaluation before requests reach providers. The Agent Gateway extends routing from model calls to agent workflows with MCP server integration. Observability integrates with Langfuse, Arize Phoenix, OpenTelemetry, and MLflow for complete request tracing. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One OpenAI-compatible endpoint, 1,600+ models behind it: Portkey Gateway sits between your app and every major LLM provider, deciding in real time where each request should go. When a provider times out or returns errors, automatic retries with exponential backoff kick in, and fallback rules quietly move traffic to your second-choice model before users notice anything. Load balancing spreads calls across multiple API keys and providers by weight, conditional routing sends each request down a different path based on its content or metadata, and per-request timeouts cut off providers that hang. Guardrails inspect inputs and outputs as traffic passes through: block banned words, screen for sensitive content, and enforce the checks your compliance team asked for, with dozens of pre-built rules ready to attach. Text is only the start: the same API shape reaches vision models, speech-to-text, text-to-speech, image generation, and realtime websocket APIs, so multimodal features stop multiplying your integration work. Routing policies live in JSON configs you attach per request or per client, which means experiments, cost controls, and provider migrations ship as configuration changes, not redeploys. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Respan Gateway is a single-binary LLM gateway that lets the SDK you already use call any AI model from any provider, translating between OpenAI, Anthropic, and Gemini API formats on the way through. Point an OpenAI, Anthropic, or Gemini client at it and you reach chat models on sixteen providers, plus any OpenAI-compatible endpoint of your own, with streaming, tools, images, and reasoning translated. Traffic control works the way operators think: name a route, send it to the cheapest or fastest provider, retry on failure, fall back to a second vendor, or hedge a slow call by racing another provider. Models that never had tools get them anyway, because MCP servers and built-in search, browsing, code, document, and memory tools run inside ordinary chat requests behind the gateway. Realtime voice crosses vendors too, with an OpenAI Realtime client able to talk to Gemini Live, alongside images, speech, transcription, embeddings, and video through one endpoint. A live request log in the web UI shows the client key, model, provider, status, tokens, cost, and latency of every call, so runaway agents and surprise bills are visible immediately. Everything is a plugin on one public Rust API, from routing and pricing to tracing, with examples for guardrails, PII masking, and billing. One TOML file configures keys, providers, routes, and tools, validates before it starts, and reloads on signal. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Your apps keep using the OpenAI SDK while Antirion, a self-hosted AI gateway, decides which provider answers, what each team may spend, and what gets logged. Clients speak the OpenAI, Anthropic, Gemini, or Bedrock APIs, and Antirion translates between them, so a Claude deployment can answer an OpenAI-style call, streams included. Add providers such as Amazon Bedrock, Vertex AI, Mistral, Groq, DeepSeek, or a vLLM server, then publish them as named Models. Routing rules send traffic by team or project with fallback chains, weighted splits, least-cost or least-latency targets, retries, hedging, and circuit breakers. Budgets and rate limits in dollars or tokens apply per organization, team, project, and key, with projections and alerts before money runs out. Guardrails check for personal data, secrets, keywords, and regex matches, then block, redact, flag, or divert. Exact and semantic caching answers repeated prompts without calling a provider. The request log records every attempt, route, token count, cost, and guardrail verdict, with Prometheus metrics and OTLP traces. An MCP gateway and A2A proxy put tools and agents behind the same keys. Sign in with OIDC, SAML, or GitHub, provision users with SCIM, and review every change in the audit log. Changes you choose can require a change request or a second person's approval, and the Playground sends requests from the console. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Juggling separate API keys, incompatible payload schemas, and unpredictable rate limits across OpenAI, Anthropic, and Google Gemini slows down every AI workflow, and Octopus solves it by consolidating all your upstream model providers into one self-hosted API gateway. Point Claude Code, OpenAI Codex CLI, or any SDK client at a single gateway URL and translate requests transparently between OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Gemini endpoints without modifying client code. Group multiple provider channels under a unified model alias so your coding agents automatically fail over to backup providers and shield active tasks from upstream HTTP errors or rate limits. Watch live end-to-end request routing on the interactive topology canvas as prompts flow from client applications through model groups to specific upstream channels. Synchronize available model catalogs directly from connected providers and pull up-to-date token pricing from models.dev, or define custom per-million input, output, and cache-read rates for private deployments. Inspect detailed request logs, monitor token consumption trends, and audit daily API spend across every channel from the web dashboard. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

Anyone who pays for Claude Code, Codex, and Gemini CLI separately ends up juggling three endpoints, three sets of keys, and three kinds of rate-limit surprise, and CliRelay collapses all of that into one self-hosted gateway with a control panel. Clients speak the OpenAI Chat or Responses, Anthropic Messages, or Gemini protocol, and CliRelay translates each request for whichever upstream account is healthy. Channel groups limit where each key may send traffic, round-robin, fill-first, and session-sticky scheduling spread it, and cooldowns with automatic failover move requests off accounts that have run out of quota. Add upstreams from the /manage console through browser sign-in, device codes, or bulk credential import, covering Claude, Codex, Gemini, Antigravity, Vertex AI, AWS Bedrock, Grok, Qwen, Kimi, and any OpenAI-compatible service. Portal accounts own client keys with daily, period, and lifetime quotas, RPM and TPM limits, and per-model pricing. A monitor center scores health and charts live traffic, P50 to P99 latency, and time to first token, while every request lands in a searchable log with tokens, latency, and cost. Tenants, roles with resource.action permissions, content moderation profiles, an outbound proxy pool, and an audit trail make shared deployments workable, and the console speaks English, Simplified Chinese, and Russian. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Instead of scattering provider credentials and vendor-specific SDK logic across dozens of microservices, engineering teams route every LLM call through GoModel, an open-source AI gateway that unifies 34 model providers behind OpenAI-compatible and Anthropic-compatible endpoints. Pointing existing OpenAI or Anthropic SDKs at the gateway unlocks chat completions, the Responses API, Anthropic Messages, embeddings, batch jobs, and realtime audio across OpenAI, Anthropic, Gemini, Bedrock, Azure, Groq, DeepSeek, Ollama, and vLLM. Platform operators publish virtual model aliases that distribute traffic via weighted round-robin or lowest-cost routing while rotating multiple API keys per provider and failing over through circuit breakers during upstream outages. Sticky session routing keeps multi-turn agent conversations on the same target to preserve warm prompt caches, while exact-match and vector semantic caching via Qdrant, pgvector, Pinecone, or Weaviate cut duplicate token costs. Versioned workflows and pre-dispatch guardrails enforce prompt injection rules, rate limits, and hard USD spend budgets across hierarchical tenant user paths before requests are dispatched. The built-in admin dashboard offers live audit logs, per-tenant cost analytics, LiteLLM key migration, an interactive request playground, Model Context Protocol tool aggregation, and OpenTelemetry and Prometheus metrics. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Bifrost is an open-source AI gateway that unifies 23+ LLM providers into a single OpenAI-compatible endpoint with automatic failover, semantic caching, and built-in cost governance, so one provider going down never takes your production AI application with it. Point your existing OpenAI or Anthropic SDK at Bifrost's local endpoint and gain access to OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Groq, Mistral, and Ollama without changing application code. Define fallback chains that automatically switch providers when one returns errors or exceeds latency thresholds, keeping response times stable during outages. The built-in web dashboard at port 8080 lets you configure providers, create virtual API keys, monitor live request traffic, and review analytics without editing configuration files. Semantic caching combines exact hash matching with vector similarity search via Weaviate, serving cached responses for identical or paraphrased prompts in sub-millisecond time to cut costs on repetitive workloads. The MCP gateway connects AI agents to external tools like filesystems, databases, and web APIs, exposing them to clients such as Claude Desktop and Cursor with per-key allow-lists. Four-tier budget hierarchy at customer, team, virtual key, and provider levels enforces spend caps, rate limits, and model restrictions across your organization. Extend functionality through custom Go plugins for analytics, monitoring, or security middleware. Native Prometheus metrics and OpenTelemetry distributed tracing give operations teams full production observability. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

New API is a Go-powered LLM gateway that funnels over 40 AI providers, from OpenAI and Anthropic to Google Gemini, Azure, AWS Bedrock, DeepSeek, and Mistral, through a single OpenAI-compatible endpoint with intelligent routing and enterprise cost controls. Bidirectional format conversion translates between OpenAI Chat Completions, Claude Messages, and Gemini GenerateContent APIs transparently, so clients call any provider using their preferred format while the gateway handles the rest. Routing logic supports weighted random channel selection, priority-based failover, and automatic retry on provider errors to maximize uptime. The React admin dashboard shows usage charts, token consumption tracking, per-model cost breakdowns, and real-time request monitoring. Multi-tenant architecture includes three role levels, token-based authentication, per-user quota management, and a three-phase billing system with tiered pricing via a custom expression language. Multimodal support covers text completion, vision, embeddings, text-to-speech, speech-to-text, image generation, and creative task providers like Midjourney-Proxy and Suno-API. The interface ships in five languages including English, Chinese, Japanese, and French. Docker deployment runs with SQLite or MySQL for persistence. 45,000+ GitHub stars. AGPL-3.0 licensed.

BitRouter is a context-aware LLM router that learns which model delivers the cheapest successful outcome per workflow step, cutting agent costs by up to 80% while maintaining 96% quality versus all-frontier baselines. Point any agent runtime at http://localhost:4356 with a one-line OPENAI_BASE_URL change and BitRouter routes to OpenAI, Anthropic, Google, Groq, DeepSeek, Mistral, Moonshot, MiniMax, Nvidia, and any OpenAI-compatible endpoint simultaneously, normalizing authentication, streaming, and cross-protocol translation between wire formats. The act-observe-evaluate-learn loop traces every hop with cost, tokens, and latency attribution, scores each decision against a versioned policy-lock.yaml, then tightens routes automatically with no LLM judge in the path. Native MCP gateway auto-discovers tools from connected servers and makes them routable and governed alongside model calls. Agent Client Protocol integration enables the TUI to manage Claude Code, Codex, OpenCode, OpenClaw, Gemini, and Copilot sessions in real time with inline tool-call approval and live streaming. Built-in guardrails inspect, redact, or block risky content before requests leave your network. Virtual keys scope API access per agent or user without exposing upstream credentials. Per-agent spend caps and loop guards contain runaway cost automatically. Multi-account failover reroutes mid-run so rate limits never re-pay completed work. Ships as a single Rust binary via npm or Cargo. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Software developers running Claude Code, Cursor, Codex, and Aider use AgentsView to trace token expenditures, search full session transcripts, and analyze multi-agent concurrency timelines. Engineers can search through historical developer sessions to locate specific code modifications, prompt chains, or debugging rationales across extensive programming workflows. The real-time activity timeline visualizes concurrent agent operations, revealing peak execution windows, task run durations, and automated tool invocations. The usage analytics engine calculates financial costs and token volumes per model by applying live pricing tables while accounting for prompt caching read discounts. Developers inspect granular turn-by-turn session transcripts complete with raw model payloads, file difference diffs, executed shell commands, and syntax errors. The recall browser extracts key technical context from past programming sessions, organizing facts into an indexed reference knowledge corpus for reuse across future project tasks. Administrators can replicate SQLite session tables into DuckDB analytics files or PostgreSQL warehouses to query longitudinal metrics using SQL analytical tools or custom reporting scripts. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Backed by Y Combinator and trusted by over 2,300 companies processing billions of observations monthly, Langfuse is the most widely adopted open-source platform for building, monitoring, evaluating, and debugging LLM applications. The hierarchical tracing engine captures every LLM call, tool invocation, retrieval step, and agent action as nested spans based on OpenTelemetry, with automatic cost calculation, latency tracking, and token usage attribution across sessions and users. Prompt Management separates prompts from code with versioned artifacts, label-based deployments, one-click rollbacks, and runtime SDK fetching with server-side caching, while linking every generation back to its exact prompt version for attribution analytics. The evaluation system supports LLM-as-a-judge scoring, heuristic code evaluators, user feedback collection, and manual annotation workflows that run automatically on production traces or against curated datasets. The Playground enables interactive prompt testing on real production inputs with side-by-side model comparison across providers. Datasets and Experiments define test cases for systematic benchmarking with comparative result visualization. Native SDKs for Python and TypeScript provide decorator-based instrumentation, while 100+ integrations cover LangChain, LlamaIndex, OpenAI SDK, LiteLLM, Vercel AI SDK, and any OpenTelemetry-instrumented framework. The analytics dashboard surfaces cost breakdowns, quality scores, latency percentiles, and usage trends across models and prompt versions. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

AI and API products live or die by usage, and OpenMeter is the open-source stack that measures it, rations it, and bills for it. Your application emits CloudEvents for every billable action; OpenMeter validates and deduplicates them, then aggregates usage in real time so dashboards and enforcement checks never wait for a nightly batch. Meters slice that usage by customer, time window, or any dimension you attach, which turns raw events into answers like tokens per workspace this month. Entitlements decide who gets what: metered allowances with balances and overage, boolean switches for features like SSO, and static JSON configuration for per-customer model access. Grants top allowances up once or on a schedule, resets roll balances over each usage period, and one API call tells your app whether a customer still has access right now. On the money side, plans carry flat, per-unit, tiered, package, or dynamic prices; subscriptions, credit balances, and the invoice lifecycle run alongside, with webhooks announcing everything to your other systems. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Your AI application is burning through API tokens faster than you can refresh the billing page, and you have no idea which prompt template is responsible. OpenLIT plugs that visibility gap with a self-hosted observability platform built specifically for LLM workloads. Add one line of code to instrument 90+ LLM providers, agent frameworks, and vector databases, then watch every request flow through a tracing dashboard that shows tokens consumed, latency measured, and dollars spent per call, per model, per environment. The requests view lists every LLM interaction with provider, model, cost, and token breakdown in a filterable table, while the trace detail panel lets you drill into individual spans to read the exact prompt sent and response received. Prompt Hub turns prompts into versioned artifacts you deploy, rollback, and A/B test without touching application code. OpenGround compares models side by side on the same input, so you can evaluate cost-versus-quality tradeoffs before committing to a provider. Automated evaluations run LLM-as-a-judge scoring on live production traces, flagging hallucinations, bias, and toxicity in real time. The Vault stores and rotates API keys centrally so secrets stay out of your codebase. Custom dashboards let you build drag-and-drop monitoring views with charts, stat cards, and tables backed by SQL queries against ClickHouse. GPU utilization, memory, temperature, and power metrics feed into the same platform for end-to-end infrastructure visibility. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.