Every call your AI agents make becomes a searchable trace in LangWatch, and the same platform tests and scores those agents before a new prompt or model reaches users. The Trace Explorer filters by origin, user, model, cost, and evaluator, reads each run as a waterfall, flame graph, topology, or sequence diagram, and takes traces from the Python, TypeScript, and Go SDKs or any OpenTelemetry source. Agent Testing sends simulated users, in text or beta voice calls, through your agent while a judge passes or fails every criterion with its reasoning, adds multi-turn red teaming, and compares two agents side by side. The Experiments workbench scores prompts and agents on datasets with LLM judges, exact match, and RAG checks, then compares runs on cost, latency, and pass rate, while online evaluations monitor live traffic and guardrails block unsafe replies. Prompts keep version history and staging or production tags, Workflows wire prompts, Python code, and HTTP calls on a visual canvas, and annotation queues route traces to human reviewers. Alerts fire when a graph crosses a threshold, automations post to Slack, email, or webhooks, and scheduled reports go out by email or Slack. Claude Code, Codex, Copilot, Gemini CLI, and opencode report session cost, pull request cost, and terminal replays, and an MCP server exposes trace search and prompt management to coding assistants. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Backed by Y Combinator and trusted by over 2,300 companies processing billions of observations monthly, Langfuse is the most widely adopted open-source platform for building, monitoring, evaluating, and debugging LLM applications. The hierarchical tracing engine captures every LLM call, tool invocation, retrieval step, and agent action as nested spans based on OpenTelemetry, with automatic cost calculation, latency tracking, and token usage attribution across sessions and users. Prompt Management separates prompts from code with versioned artifacts, label-based deployments, one-click rollbacks, and runtime SDK fetching with server-side caching, while linking every generation back to its exact prompt version for attribution analytics. The evaluation system supports LLM-as-a-judge scoring, heuristic code evaluators, user feedback collection, and manual annotation workflows that run automatically on production traces or against curated datasets. The Playground enables interactive prompt testing on real production inputs with side-by-side model comparison across providers. Datasets and Experiments define test cases for systematic benchmarking with comparative result visualization. Native SDKs for Python and TypeScript provide decorator-based instrumentation, while 100+ integrations cover LangChain, LlamaIndex, OpenAI SDK, LiteLLM, Vercel AI SDK, and any OpenTelemetry-instrumented framework. The analytics dashboard surfaces cost breakdowns, quality scores, latency percentiles, and usage trends across models and prompt versions. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Agenta delivers a comprehensive open-source LLMOps workspace that covers the full lifecycle of AI application development — from prompt engineering through production monitoring. The platform supports 15+ model providers including OpenAI, Anthropic, Google Gemini, Mistral, Groq, Together AI, Azure, AWS Bedrock, and self-hosted models via Ollama, enabling teams to switch between providers without code changes. The prompt playground allows side-by-side comparison of different configurations, while the evaluation system offers LLM-as-a-Judge assessment, 20+ pre-built evaluators covering semantic similarity, regex matching, and factual accuracy, plus custom Python evaluators for domain-specific requirements. Teams run evaluations through both the web UI for subject matter experts and the Evaluation SDK for programmatic CI/CD integration. The observability layer captures full trace visibility across complex agentic workflows, flagging quality issues like hallucinations and off-topic responses in real time. Human annotation workflows let domain experts review and annotate LLM outputs, feeding corrections back into the evaluation loop. The architecture supports Chain of Prompts, RAG pipelines, and multi-step agent workflows, integrating with frameworks like LangChain and LlamaIndex. Self-hosting deploys via Docker Compose with Traefik for routing, requiring only a clone, environment configuration, and a single docker compose command. On RepoCloud, deploy Agenta on a dedicated VPS with root SSH access, persistent storage for evaluation datasets and traces, and complete control over model provider credentials, all under the MIT license with no usage restrictions.

Backed by Y Combinator (S24) and processing traces from thousands of AI agents in production, Laminar is the open-source observability platform that treats agent debugging as a first-class engineering discipline rather than an afterthought. Its OpenTelemetry-native SDK auto-instruments Vercel AI SDK, LangChain, OpenAI, Anthropic, Gemini, Browser Use, Stagehand, Mastra, Pydantic AI, and the OpenAI Agents SDK with a single line of code, capturing every LLM turn, tool call, and sub-agent delegation as nested spans with full input/output data and token costs. The Signals engine lets you describe failures in plain language — "agent is stuck in a loop" or "tool returned empty results" — then reads every trace and alerts via Slack when it detects a match. A built-in debugger records runs and replays them from cache so each iteration takes seconds, designed for Claude Code, Cursor, or Codex to drive the repair loop via the MCP server or CLI. Run code-first evaluations in Python or TypeScript locally or in CI/CD pipelines, build datasets from production traces, and query everything with raw SQL through custom dashboards, the in-app editor, or your coding agent. The Rust backend delivers 20x trace compression, a custom real-time streaming engine, ultra-fast full-text search, and gRPC ingestion, while ClickHouse powers columnar analytics and PostgreSQL stores application state. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Your AI application is burning through API tokens faster than you can refresh the billing page, and you have no idea which prompt template is responsible. OpenLIT plugs that visibility gap with a self-hosted observability platform built specifically for LLM workloads. Add one line of code to instrument 90+ LLM providers, agent frameworks, and vector databases, then watch every request flow through a tracing dashboard that shows tokens consumed, latency measured, and dollars spent per call, per model, per environment. The requests view lists every LLM interaction with provider, model, cost, and token breakdown in a filterable table, while the trace detail panel lets you drill into individual spans to read the exact prompt sent and response received. Prompt Hub turns prompts into versioned artifacts you deploy, rollback, and A/B test without touching application code. OpenGround compares models side by side on the same input, so you can evaluate cost-versus-quality tradeoffs before committing to a provider. Automated evaluations run LLM-as-a-judge scoring on live production traces, flagging hallucinations, bias, and toxicity in real time. The Vault stores and rotates API keys centrally so secrets stay out of your codebase. Custom dashboards let you build drag-and-drop monitoring views with charts, stat cards, and tables backed by SQL queries against ClickHouse. GPU utilization, memory, temperature, and power metrics feed into the same platform for end-to-end infrastructure visibility. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Software developers running Claude Code, Cursor, Codex, and Aider use AgentsView to trace token expenditures, search full session transcripts, and analyze multi-agent concurrency timelines. Engineers can search through historical developer sessions to locate specific code modifications, prompt chains, or debugging rationales across extensive programming workflows. The real-time activity timeline visualizes concurrent agent operations, revealing peak execution windows, task run durations, and automated tool invocations. The usage analytics engine calculates financial costs and token volumes per model by applying live pricing tables while accounting for prompt caching read discounts. Developers inspect granular turn-by-turn session transcripts complete with raw model payloads, file difference diffs, executed shell commands, and syntax errors. The recall browser extracts key technical context from past programming sessions, organizing facts into an indexed reference knowledge corpus for reuse across future project tasks. Administrators can replicate SQLite session tables into DuckDB analytics files or PostgreSQL warehouses to query longitudinal metrics using SQL analytical tools or custom reporting scripts. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

To pull a team's skills, MCP servers, hooks, prompts, and agent files out of scattered Git repositories, Observal keeps them in one reviewed registry and traces real sessions to show which ones help. Build an agent visually, with a live config preview for every harness. Submissions wait in an admin review queue, where reviewers inspect side-by-side version diffs before approving a release. Teamspaces give each team its own namespace, and approved releases can be forked into independent drafts. One pull command then writes the right config for Claude Code, Cursor, Kiro, Codex, Copilot, Pi, OpenCode, Antigravity, or Goose. Every approved resource is discoverable through the open Agentic Resource Discovery standard, and a running agent can hand part of its task to another approved agent or a remote Agent2Agent service. Hooks record each session as a trace of prompts, tool calls, MCP round trips, tokens, and errors, with turn-by-turn replay for debugging and audits. Insight reports, generated through providers such as Anthropic, OpenAI, or Ollama, list what is working, what is hindering, and quick wins. A dashboard follows adoption, cost, and department usage, a leaderboard ranks agents by downloads, and audit logs, SAML single sign-on, and SCIM provisioning are included. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

CPA Manager Plus is a self-hosted observability dashboard and management panel that tracks every AI request flowing through your CLI Proxy API gateway, breaking down failures, costs, and account health across providers like OpenAI, Anthropic, xAI, and Codex in one interface. When a request fails, drill into the persistent history to see status codes, affected models, latency, and redacted failure evidence without exposing raw response bodies. The cost analytics engine breaks down token consumption and estimated spend by model, provider, account, API key, project, channel, and time range while tracking input, output, reasoning, cache, and service-tier pricing semantics separately. Model prices sync automatically from models.dev with LiteLLM and OpenRouter fallbacks, and you can add local overrides for aliases or internal models. For teams running Codex or xAI accounts, the health inspector reads quota windows, reset evidence, credential state, and workspace status on a configurable schedule, routing credential failures into an action queue for review rather than letting them silently degrade throughput. Deploy the Lightweight Panel to replace your existing CPA management UI without adding another service, or run Full Mode as a single Docker container that adds the Manager Server with persistent SQLite storage for request history, historical analytics, and automated account inspections. Export or import request history as JSONL for external analysis, and back up the SQLite files alongside your encrypted management keys. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Trusted by thousands of organizations with over 30 million monthly downloads and 20,000+ GitHub stars, MLflow is the largest open-source AI engineering platform providing end-to-end lifecycle management for traditional ML models, LLMs, and AI agents. The OpenTelemetry-based tracing system captures complete request flows through any LLM provider or agent framework — including OpenAI, LangChain, DSPy, Vercel AI, PydanticAI, and smolagents — with one-line auto-instrumentation that tracks inputs, outputs, token usage, and costs at every intermediate step. MLflow's evaluation engine offers 50+ built-in metrics and LLM judges for systematic quality assessment, detecting issues across correctness, latency, adherence, relevance, and safety dimensions before code reaches production. The Prompt Registry versions, tests, and deploys prompts with full lineage tracking while automated optimization algorithms improve prompt performance using evaluation feedback. The AI Gateway provides a unified API endpoint for all LLM providers, enforcing rate limits, cost controls, and access policies across the organization. MLflow 3.0 introduces the LoggedModel abstraction linking traces, metrics, and prompts to specific model versions across Python, TypeScript, Java, and R SDKs. The model registry manages deployment workflows with automated quality gates, while experiment tracking records parameters, metrics, and artifacts across training runs. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache License 2.0 licensed.

With 20,500+ GitHub stars and over 6,000 organizations running it in production — including a Fortune 100 company ingesting more than 4 PB per day — OpenObserve is the open-source observability platform that replaces your entire Datadog, Splunk, or ELK stack with a single Rust binary deploying in under two minutes. Apache Parquet columnar storage with zstd compression on S3-compatible object storage delivers 140x lower storage costs than Elasticsearch while providing better query performance on a quarter of the hardware. Ingest logs, metrics, and distributed traces via native OpenTelemetry OTLP endpoints with no vendor lock-in. Query logs and traces with standard SQL, metrics with SQL or PromQL — no proprietary query language to learn. The built-in dashboard builder offers 19 chart types including time-series graphs, heatmaps, gauges, tables, and top-K lists with drag-and-drop layout combining data from all signal types. Data pipelines process, enrich, redact, or normalize ingestion streams using Vector Remap Language for real-time transformations including PII redaction and logs-to-metrics conversion. Real User Monitoring captures frontend performance with session replay. The Service Catalog provides topology-based trace analysis with side-panel drill-downs into database queries and error details. Alerting supports real-time and scheduled rules with SQL and PromQL conditions. Native multi-tenancy isolates organizations and streams with complete data separation. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPLv3 licensed.

Created by Uber Technologies and graduated as the seventh CNCF top-level project in October 2019 with over 23,000 GitHub stars, Jaeger has become one of the most widely deployed open-source distributed tracing platforms, processing billions of spans per day in production environments at organizations including Uber, Red Hat, and Shopify. Version 2 rebuilt the platform on the OpenTelemetry Collector framework, inheriting its extensible pipeline architecture while implementing Jaeger-specific features as extensions and components, enabling seamless integration with the OpenTelemetry ecosystem through native OTLP protocol support. The platform stores traces in Cassandra 4.0+, Elasticsearch 7.x/8.x, OpenSearch 1.0+, ClickHouse, or the embedded Badger database for development setups. Three sampling strategies control trace volume: head-based sampling with constant, probabilistic, and rate-limiting modes, tail-based sampling using the OpenTelemetry Collector processor that evaluates complete traces before storage decisions, and adaptive sampling that dynamically adjusts probabilities based on observed traffic patterns. Service Performance Monitoring computes RED metrics directly from spans, displaying request rates, error rates, and latency percentiles in the Monitor tab with drill-down from aggregate service views to individual traces. The web UI provides trace search with multi-field filtering, trace detail views with span timeline visualization, trace comparison across services, and dependency graphs mapping service relationships from actual traffic. Deployment options range from a single all-in-one binary for development to distributed collector-ingester-query configurations with Kafka intermediate buffering for production scale. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

With over 31,000 GitHub stars and native OpenTelemetry support that eliminates vendor lock-in from day one, SigNoz delivers full-stack observability covering metrics, traces, and logs in a single pane of glass without the per-host pricing model of commercial APM platforms. The platform ingests telemetry data through the OpenTelemetry Collector, supporting auto-instrumentation for Java, Python, Node.js, Go, Ruby, PHP, and .NET applications with zero code changes required for basic tracing. ClickHouse serves as the columnar storage backend, providing fast aggregation queries over billions of spans and log lines with configurable retention policies and tiered storage. The distributed tracing view renders flame graphs and Gantt charts showing request flow across microservices with latency breakdowns, error rates, and p99 percentile calculations. Custom dashboards support PromQL and ClickHouse SQL queries with time-series charts, bar graphs, tables, and value widgets. The log management pipeline supports structured and unstructured logs with full-text search, log pipelines for parsing and enrichment, and correlation with traces via trace IDs. Alert rules can be configured on any metric or log query with notification channels including Slack, PagerDuty, OpsGenie, webhooks, and email. The exceptions monitoring module automatically groups and tracks application errors with stack traces, occurrence counts, and first-seen timestamps. Service maps visualize inter-service dependencies with real-time latency and error rate overlays. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed with an enterprise edition available.

GlitchTip speaks Sentry's protocol without Sentry's operational weight - open-source error tracking that your existing SDKs already understand. The pitch is pragmatic: instrument your application with the official Sentry SDKs you already know - any language they cover - and point the DSN at your own GlitchTip instance instead. Errors, exceptions, log messages, and Content Security Policy violations flow into one place for triage, grouped into issues with stack traces, with alerts delivered by email or webhook the moment things break. Where self-hosted Sentry has ballooned into a docker-compose stack of twenty-plus containers, GlitchTip is a deliberately lean Django and PostgreSQL application a small team can actually run. Beyond errors, it bundles three more monitoring concerns: performance monitoring takes a works-out-of-the-box approach - no dashboard building, just your slowest web requests, database queries, and transactions surfaced automatically; uptime monitoring pings your sites and alerts on failures, or runs in reverse as a dead-man's-switch heartbeat for cron jobs that must check in on schedule; and log search puts application logs alongside errors for faster debugging. Unlimited projects and team members, MIT-licensed, built by Burke Software - your event volume is limited only by your own hardware.

Backed by 44,000 GitHub stars and trusted by over four million developers, Sentry is the debugging platform that captures errors, traces, replays, profiles, and metrics from your applications and connects them all through distributed tracing. The error tracking engine captures full stack traces with source context, breadcrumbs, and automatic demangling for native crashes, while intelligent grouping consolidates duplicate events into actionable issues with regression detection and automatic assignment. Performance monitoring instruments your frameworks automatically, capturing every database query, API call, cache hit, and queue operation as spans within distributed traces that flow across frontend, backend, and mobile boundaries. Session Replay produces video-like recordings of real user sessions showing DOM interactions, network requests, console messages, and errors on a synchronized timeline, with AI-powered summaries that describe what happened without watching the full replay. Continuous profiling captures CPU execution data at the function and line level for Node.js, Python, iOS, and Android, linking slow spans directly to the exact code responsible. Cron monitoring tracks scheduled jobs for failures, missed runs, and duration anomalies. The alerting engine fires notifications through Slack, PagerDuty, Opsgenie, and webhooks on new issues, regressions, error spikes, or when latency and crash-free session rate thresholds are crossed. Self-hosted deployment runs as a Docker Compose stack with PostgreSQL, ClickHouse, Kafka, Redis, and Relay. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. FSL licensed.

With 16,100 GitHub stars and created by Solomon Hykes (co-founder of Docker), Dagger eliminates proprietary YAML DSLs from CI/CD by letting developers write pipelines as real programs in their preferred language — then executing them identically on a laptop, in GitHub Actions, in GitLab CI, or on any machine with a container runtime. The BuildKit-based Dagger Engine runs every pipeline operation inside OCI containers, constructing a directed acyclic graph where each node is cached by default, parallelized automatically, and produces bit-for-bit reproducible outputs regardless of execution environment. Native SDKs generated from the GraphQL API schema provide Go, Python, TypeScript, PHP, Java, .NET, Elixir, and Rust developers with full type safety, IDE autocomplete, and the ability to unit-test CI pipelines using the same testing frameworks as application code. The module ecosystem enables cross-language composition where a Python team can invoke a Go team's build functions without learning Go, while filesystems, secrets, git repositories, and network tunnels are passed between functions as strongly-typed objects. Pipeline operations cache at container-layer granularity with content-addressed deduplication, and the interactive REPL enables step-by-step pipeline debugging with live container inspection. Host dependencies are explicit and strictly typed, eliminating implicit environment coupling that causes "works on my machine" failures. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

With 7,300+ GitHub stars and active releases through version 3.17, Woodpecker CI is the community-driven fork of Drone CI that kept the entire platform open source under Apache 2.0 — delivering a lightweight, container-native CI/CD engine that runs on minimal resources while providing enterprise-grade pipeline automation. Every pipeline step executes in an isolated Docker container, ensuring reproducible builds with zero host contamination and clean teardown after each run. Define workflows in simple YAML configuration files with step dependencies, conditional execution via when filters, service containers for databases and caches, and matrix builds that test across multiple language versions, platforms, or database engines simultaneously. Deep forge integration connects to GitHub, GitLab, Gitea, Forgejo, and Bitbucket via OAuth and webhooks, triggering pipelines on push, pull request, tag, deployment, and cron events. The plugin ecosystem inherits compatibility with the Drone plugin library — over 100 plugins for Docker builds, S3 uploads, SSH deployment, Slack and Telegram notifications, Helm deployments, and more. The server-agent architecture separates the web UI, API, and scheduler from pipeline execution, enabling horizontal scaling by adding agents across multiple machines or architectures including ARM. Secrets management supports global, organization, and repository-level scopes with approval workflows for forked repository pipelines. The admin web UI provides user management, agent monitoring, and queue control. Docker Compose deployment provisions the complete stack in under five minutes. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.