Deploy Now

SaaS Alternative

Helicone Portkey LiteLLM

Categories

Self-Hosted Developer Tools LLM Machine Learning MCP API Gateway AI Gateway

Stars

7,873

Forks

1,177

Watchers

32

Developer links

Last Updated: 09-08-2026
Share on X

Bifrost

Bifrost is an open-source AI gateway that unifies 23+ LLM providers into a single OpenAI-compatible endpoint with automatic failover, semantic caching, and built-in cost governance, so one provider going down never takes your production AI application with it. Point your existing OpenAI or Anthropic SDK at Bifrost's local endpoint and gain access to OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Groq, Mistral, and Ollama without changing application code. Define fallback chains that automatically switch providers when one returns errors or exceeds latency thresholds, keeping response times stable during outages. The built-in web dashboard at port 8080 lets you configure providers, create virtual API keys, monitor live request traffic, and review analytics without editing configuration files. Semantic caching combines exact hash matching with vector similarity search via Weaviate, serving cached responses for identical or paraphrased prompts in sub-millisecond time to cut costs on repetitive workloads. The MCP gateway connects AI agents to external tools like filesystems, databases, and web APIs, exposing them to clients such as Claude Desktop and Cursor with per-key allow-lists. Four-tier budget hierarchy at customer, team, virtual key, and provider levels enforces spend caps, rate limits, and model restrictions across your organization. Extend functionality through custom Go plugins for analytics, monitoring, or security middleware. Native Prometheus metrics and OpenTelemetry distributed tracing give operations teams full production observability. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Bifrost
Bifrost
Bifrost
Bifrost
Bifrost
Bifrost
Bifrost

Benefits

  • Multi-Provider Failover Protection
  • Define fallback chains across 23+ AI providers including OpenAI, Anthropic, and AWS Bedrock. Bifrost automatically switches to backup models when one fails or exceeds latency thresholds, maintaining zero downtime for production applications.
  • Semantic Response Caching
  • Dual-layer caching combines exact hash matching with vector similarity search via Weaviate. Semantically similar prompts return cached responses in sub-millisecond time, cutting API costs on repetitive LLM workloads.
  • Centralized MCP Tool Gateway
  • Connect AI agents to filesystems, databases, and web APIs through the Model Context Protocol. Expose registered tools to Claude Desktop and Cursor with per-key allow-lists and configurable approval policies.
  • Hierarchical Budget Controls
  • Enforce spend caps, rate limits, and model restrictions at customer, team, virtual key, and provider levels. Track costs with native Prometheus metrics and OpenTelemetry distributed tracing.

Features

  • Provider Routing
  • Route requests to OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Groq, Mistral, and Ollama through a single OpenAI-compatible API endpoint with zero code changes.
  • Web Configuration Dashboard
  • Configure providers, manage virtual API keys, monitor live request traffic, and view analytics through the built-in web interface at port 8080.
  • Custom Plugin System
  • Load custom Go plugins at startup for analytics hooks, monitoring integrations, security validators, and business logic middleware using an extensible architecture.
  • Multimodal Streaming
  • Process text, images, audio, and streaming responses across all supported providers behind a unified interface with automatic format translation.
  • GitOps Deployment
  • Deploy with a declarative config.json file for CI/CD pipelines and multi-node setups with Docker, Kubernetes Helm charts, or direct VM installation.