Pagesift converts messy web pages, PDFs, and Office documents into clean Markdown and structured JSON that language models can actually use, from one self-hosted binary you run yourself. Call it the way you already call Firecrawl: the HTTP API speaks the Firecrawl v1 and v2 formats, so existing scrape, crawl, and batch jobs point at your server instead, and a Jina Reader style endpoint answers simple /r/ URL requests. Scrape a single page, submit batches, or crawl whole sites with sitemap support, focused crawling by query, robots.txt respect, and per-host rate limits by default. Headless rendering kicks in only when a page needs JavaScript, keeping plain pages fast. Documents work offline too: Word, Excel, PowerPoint, EPUB, and CSV files become Markdown with titles, links, images, and publication metadata attached. Jobs persist so long crawls survive restarts, webhooks report completions, and per-API-key limits and quotas keep multi-tenant use civilized, with Prometheus metrics and a health endpoint for operators. AI agents can drive the same engine directly through the bundled MCP server over stdio or HTTP, while developers get Python, Node, and Rust libraries. Memory stays bounded under load by design, sized from container limits rather than hope. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

More than 70 automated checks sniff out broken links, redirect chains, and missing meta tags across your whole site, and SEOnaut, the self-hosted SEO audit crawler, sorts every finding into critical, alert, or warning so you know what to fix first. Add a website as a project, press Crawl Now, and watch URLs stream in with their status codes; a Stop button ends any run that wanders off. Reports also cover duplicate titles and descriptions, heading order, hreflang, canonicals, image alt text, thin content, orphan pages, and non-HTTPS URLs, while a passed-checks list shows what is already healthy. Dashboard charts follow crawl history, issue trends, status codes, link types, and page depth. Crawl options are set per project: ignore robots.txt, follow nofollow links, include noindex pages, read the sitemap, allow subdomains, check external links, or authenticate with HTTP Basic to audit a password-protected staging site. Page Details opens any URL with its inlinks, outgoing links, redirections, and images, and URL Explorer searches the crawled pages by term. Export sitemap.xml, CSV lists, or a replayable WACZ archive. Each person picks a light or dark theme and one of five interface languages. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Eliminate indexing blind spots across fragmented company networks with Fess, a distributed enterprise search engine that crawls internal document repositories, corporate intranets, and cloud services to provide employees with instant, permission-aware file discovery. Organizations can schedule automated crawlers across web pages, network shares, databases, and third-party storage platforms to extract text and generate visual thumbnails from PDFs, spreadsheets, presentations, and compressed archives. Users can search indexed records using natural language queries, filtering results by file metadata, creation dates, categories, and geographic coordinates while viewing contextual snippets with highlighted keyword matches. Granular role-based permissions mirror directory security structures from Active Directory and LDAP so staff only see search results for documents they have explicit authorization to inspect. The browser-based administrative console lets operations teams define path mapping rules, manage virtual hosts, monitor active crawl jobs, inspect error logs, and configure scheduled re-indexing workflows without modifying server configuration files. Security teams can establish single sign-on across the organization using SAML or OpenID Connect authentication providers to secure administrative workflows and search portals. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.