Stars
Forks
Watchers
Developer links
Forge
Forge intercepts failing LLM tool calls and fixes them before they derail your agent workflow, applying rescue parsing, retry nudges, response validation, and step enforcement between your AI clients and local model backends. The proxy server mode drops in as a transparent intermediary speaking both the OpenAI chat-completions API and the Anthropic Messages API, so tools like Aider, Claude Code, Continue, and opencode connect through it without configuration changes. Under the hood, the WorkflowRunner provides a complete agentic loop manager with system prompt injection, tool execution, context compaction with configurable thresholds, and VRAM budgeting for consumer GPUs with 12-32 GB. SlotWorker enables priority-queued access to shared inference slots with automatic preemption for multi-agent architectures. The guardrails middleware exposes a two-method check-and-record API that wraps into any existing orchestration loop, providing malformed tool-call rescue parsing, retry nudge generation, required step enforcement, and prerequisite ordering without taking over execution control. Backend adapters support generic OpenAI-compatible endpoints, Ollama, llama-server, Llamafile, vLLM, and Anthropic with automatic model discovery and health checking. Architecture Decision Records document every design choice. Launched February 2026, already at 2,200+ GitHub stars. MIT licensed.
Benefits
- Drop-In LLM Reliability Proxy
- Transparent proxy server speaks OpenAI and Anthropic APIs, applying guardrails between any client and your local model server without requiring configuration changes in existing tools.
- Rescue Malformed Tool Calls
- Rescue parsing recovers tool calls from malformed JSON, missing brackets, and truncated responses that would normally cause agent loops to fail on smaller self-hosted models.
- VRAM-Aware Context Compaction
- Automatic context window management with configurable compaction thresholds and VRAM budgeting keeps multi-step workflows running within consumer GPU memory limits of 12-32 GB.
- Multi-Backend Adapter System
- Backend adapters support Ollama, vLLM, llama-server from llama.cpp, Llamafile, generic OpenAI-compatible endpoints, and Anthropic with automatic model discovery and health checking.
Features
- WorkflowRunner Agent Loops
- Full lifecycle management with system prompts, tool execution, context compaction, step enforcement, cancellation, streaming, and on_message observability callbacks.
- SlotWorker GPU Sharing
- Priority-queued access to shared GPU inference slots with automatic preemption enables multi-agent architectures sharing a single consumer GPU.
- Step Enforcement System
- Required steps, prerequisites, and terminal tool declarations constrain agent workflows with configurable retry limits and premature completion prevention.
- Guardrails Middleware API
- Two-method check and record facade wrapping ResponseValidator, StepEnforcer, and ErrorTracker integrates reliability logic into any existing orchestration loop.
- Proxy Usage Monitoring
- The proxy exposes a usage endpoint tracking token counts, tool call statistics, retry rates, and guardrail intervention metrics for the last completed request.