Stars
Forks
Watchers
Developer links
OpenLIT
Your AI application is burning through API tokens faster than you can refresh the billing page, and you have no idea which prompt template is responsible. OpenLIT plugs that visibility gap with a self-hosted observability platform built specifically for LLM workloads. Add one line of code to instrument 90+ LLM providers, agent frameworks, and vector databases, then watch every request flow through a tracing dashboard that shows tokens consumed, latency measured, and dollars spent per call, per model, per environment. The requests view lists every LLM interaction with provider, model, cost, and token breakdown in a filterable table, while the trace detail panel lets you drill into individual spans to read the exact prompt sent and response received. Prompt Hub turns prompts into versioned artifacts you deploy, rollback, and A/B test without touching application code. OpenGround compares models side by side on the same input, so you can evaluate cost-versus-quality tradeoffs before committing to a provider. Automated evaluations run LLM-as-a-judge scoring on live production traces, flagging hallucinations, bias, and toxicity in real time. The Vault stores and rotates API keys centrally so secrets stay out of your codebase. Custom dashboards let you build drag-and-drop monitoring views with charts, stat cards, and tables backed by SQL queries against ClickHouse. GPU utilization, memory, temperature, and power metrics feed into the same platform for end-to-end infrastructure visibility. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.
Benefits
- Full-Stack LLM Observability
- Trace every LLM call from prompt to response with OpenTelemetry-native instrumentation. Monitor tokens, latency, cost, and errors across 90+ providers and frameworks in a single dashboard.
- Cost Visibility Across Models
- Track spend per model, per provider, per environment, and per request. Built-in pricing for popular models plus custom pricing files for fine-tuned deployments ensure accurate budget forecasting.
- Production Quality Gates
- Automated LLM-as-a-judge evaluations score live traces for accuracy, relevance, hallucination, bias, and toxicity. Offline evaluations run the same checks in CI pipelines before deployment.
- Prompt Versioning and Deployment
- Prompt Hub manages prompts as versioned artifacts with full change history. Deploy to any environment without code changes and roll back instantly when output quality drops.
Features
- LLM Tracing
- Instrument once with vendor-neutral SDKs for Python, TypeScript, or Go. Every LLM call, tool invocation, and retrieval produces OpenTelemetry traces filterable by user, session, or metadata.
- OpenGround Playground
- Compare LLM outputs side by side across OpenAI, Anthropic, Google AI, Cohere, DeepSeek, and 50+ providers. Evaluate cost, latency, and output quality on the same prompt before choosing a model.
- Custom Dashboards
- Build drag-and-drop dashboards with bar charts, line charts, stat cards, and interactive tables. Queries run against ClickHouse with dynamic filtering and time range controls.
- Secrets Vault
- Store and rotate LLM API keys centrally with AES-256 encryption. Applications retrieve keys at runtime; no restart or code change needed when credentials rotate.
- GPU Monitoring
- Collect utilization, memory, temperature, and power metrics from NVIDIA, AMD, and Intel GPUs via OpenTelemetry. Correlate GPU performance with LLM inference latency in unified dashboards.