Complete log pipeline solution in Python. Runs fast in Go.

vaerohq vaerohq Last update: Dec 31, 2023

Vaero Logo

Go Contributions

View the documentation and quickstart guide online for usage information.

What is Vaero?

Vaero is a programmable high-performance data pipeline for log data. Collect, transform, and route your log data from any log source to any destination. Vaero helps you save on log storage costs, send data to different systems of analysis, normalize data to different formats, enrich data with more information, and control logs from all your vendors in one place.

Principles

  • Easy to program - Code your log pipelines in Python. No need to learn a config file format or new language.
  • Fast - Log pipelines are executed entirely in Go, with performance in mind.
  • Building blocks - Built-in transforms and source and destination integrations, so you don't have to reinvent the wheel.

Use Cases

  • Reduce log storage costs by filtering and routing logs
  • Centralize log management instead of configuring multiple sources
  • Adjust one system when you change vendors, not all your sources
  • Normalize log data between systems
  • Mask PII before storage
  • Improve overall log ETL performance

Code Log Pipelines in Python

from vaero.stream import Vaero

result = Vaero().source("http_server", port = 8080, endpoint = "/log") \
        .rename("hostname", "host") \
        .add("newfield", "Hello, world!") \
        .sink("s3")

Vaero.start()

Vaero's Python syntax is modern and easy to use. Log pipelines are executed with high-performance Go code.

Getting Started

Pipelines

Deployment

Adminstration

Reference

Configuration


Built and supported by Vaero

PRAGMA foreign_keys = off; BEGIN TRANSACTION; COMMIT TRANSACTION; PRAGMA foreign_keys = on;

Subscribe to our newsletter