Deploy Now

Stars

4,104

Forks

504

Watchers

106

Developer links

YaCy

Search engines normally rent you answers from an index you cannot inspect; YaCy installs the entire machine on your server: the crawler, the index, the ranking, and the search page itself. Run it as a public web search portal with faceted results, topic clouds, and image search, or point the crawler at your intranet where a network scanner discovers HTTP, FTP, and file servers for a private enterprise search. Expert crawl jobs take start URLs, sitemaps, or link lists with depth limits, per-domain caps, and freshness rules, and the scheduler keeps re-crawling so results stay current. Peers exchange index data over a built-in peer-to-peer protocol, so a fresh installation can answer from the shared network index before its own crawl has seen much of the web. Private modes cut that connection for local-only answers computed on your hardware alone. The index browser walks stored hosts and draws link graphs of what the crawler found. Every page in the interface has an XML or JSON twin for automation, and the portal design tools restyle the result page for embedding in your own site. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. GPL-2.0 licensed.

YaCy
YaCy
YaCy
YaCy
YaCy

Benefits

  • Search Without A Middleman
  • Queries run against an index on your own hardware instead of a remote advertising platform, so nothing about what you search leaves the server unless you choose the shared peer network.
  • A Crawler That Keeps Up
  • Expert crawl jobs accept start URLs, sitemaps, and link lists with depth and per-domain limits, and scheduled re-crawls apply freshness rules so the index tracks changing sites automatically.
  • Intranet Search Appliance
  • A network scanner discovers HTTP, FTP, and SMB servers on your LAN, and the crawler indexes them into a private search portal that replaces commercial enterprise search appliances.
  • Portal Ready And API Driven
  • The result page can be restyled and embedded in your own web applications, and every interface page exposes an XML or JSON twin, so searches and administration can be automated.

Features

  • Peer-To-Peer Index Exchange
  • Peers swap index data over a built-in protocol, giving a new installation access to the shared network index, with opt-out modes for private local-only search.
  • Production Web Crawler
  • The crawler handles start URLs, sitemaps, media wikis, forums, and proxy scraping with depth limits, domain caps, robots rules, and scheduled freshness checks.
  • Faceted Search Portal
  • Search results carry provider, author, filetype, and language facets, topic tag clouds, date navigation, and image search, all served from your own portal page.
  • Index Browser Visualizations
  • The index browser walks stored hosts and documents, shows pending and failed loads, and renders link structure graphs of crawled sites.
  • Ranking And Portal Design
  • Ranking heuristics and the result page layout are configurable in the administration interface, including which navigators, previews, and metadata each result shows.
  • HTTP APIs Everywhere
  • Pages across the interface expose XML and JSON versions marked by the API icon, and shell scripts in the distribution wrap common administration calls.

Apps Similar to YaCy