Categories
Dashboards Industrial IoT Data Pipelines Stream Processing Time Series OPC UA Manufacturing AnalyticsStars
Forks
Watchers
Developer links
Apache StreamPipes
On the factory floor, the data already exists; it is just locked inside machines, PLCs, and protocols nobody on the analytics team speaks. StreamPipes unlocks it with guided adapters for OPC UA (including a web-based node browser), MQTT, Kafka, Pulsar, REST services, and industrial controllers, turning each source into a live stream in minutes. Domain experts then drag those streams into the pipeline editor, chain processors and sinks on a canvas, and watch connection validation check schema compatibility as they build, with live preview showing real values before anything is saved. Pipelines filter, enrich, aggregate, and route machine data, raise alerts, and write results to the built-in time-series data lake, where the data explorer charts history and spots trends. Finished pipelines run under an overview that starts, stops, and bulk-manages them, while detail views expose logs, metrics, status, and the pipeline as code. Live dashboards carry the same data to shop-floor screens and operations centers, organized around assets so a line, a cell, or a single machine becomes the navigation unit. Developers extend everything with custom adapters, processors, and sinks built on the Java SDK and deployed as microservices, or drive the platform from Python, Go, and Java clients. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.
Benefits
- Machines Connected Without Middleware
- Guided adapters for OPC UA, MQTT, Kafka, Pulsar, REST, and PLCs, with a web-based OPC UA node browser, turn industrial equipment into live streams without custom protocol code or integration projects.
- Pipelines Built Visually, Validated Live
- The drag-and-drop editor checks schema compatibility at every connection and previews real data while you build, so filtering, enrichment, and alerting logic is verified before a pipeline ever runs.
- History And Dashboards Included
- A built-in time-series data lake stores historical values for charting and trend analysis, while live dashboards publish the same streams to shop-floor screens and operations centers.
- Extensible By Your Developers
- Custom adapters, processors, and sinks built with the Java SDK run as microservices, and Python, Go, and Java clients let existing applications consume streams and manage pipelines programmatically.
Features
- StreamPipes Connect Adapters
- A guided setup flow connects OPC UA servers, MQTT brokers, Kafka and Pulsar topics, REST endpoints, and PLCs, with schema proposals generated from live sample data.
- Visual Pipeline Editor
- Streams, processors, and sinks snap onto a canvas with automatic layout, in-context configuration against upstream schemas, validation-aware saving, and an exportable code view.
- Pipeline Operations Overview
- All pipelines list their running state in one table with start, stop, and bulk actions, while detail pages add logs, metrics, status history, and structural inspection.
- Data Lake And Explorer
- Historical measurements land in a time-series store that the explorer charts with raw-data previews, table coloring, and visualizations tailored to industrial time-series analysis.
- Assets And Live Dashboards
- Pipelines, streams, and dashboards attach to assets representing lines, cells, or machines, and dashboard widgets render real-time values for monitoring walls.