Blog

Updates from the Arroyo team

Announcing Arroyo 0.2.0

Arroyo 0.2.0 brings a number of improvements including native Kubernetes support, new SQL features, and many other fixes and improvements.

Micah Wylde

Micah Wylde

CEO of Arroyo

The Kit Fox lives in the deserts of the American Southwest. They commonly grow to 3-6 pounds and can reach speeds up to 40 MPH.

We're excited to announce the 0.2.0 release of Arroyo, our first since we open-sourced the project last month. Arroyo is a state-of-the-art stream processing engine that lets anyone build complex real-time data pipelines with SQL.

With the 0.2.0 release, we are continuing to push forward on features, stability, and productionization. We've added native Kubernetes support and easy deployment via a Helm chart, expanded our SQL support with features like JSON functions and windowless joins, and made many more fixes and improvements detailed below.

We are also thrilled to welcome three new contributors to the project:

Looking forward to the 0.3.0 release, we will continue to improve our SQL support with the ability to create sources and sinks directly as SQL tables, views, UDFs and external joins. We will also be adding a native Pulsar connector and making continued improvements in performance and reliability.

Excited to be part of the future of stream processing? Come chat with the team on our discord, check out a starter issue and submit a PR, and let us know what you'd like to see next in Arroyo!

Features

Native Kubernetes support

As of release 0.2.0, Arroyo can natively target Kubernetes as a scheduler for running pipelines. We now also support easily running the Arroyo control plane on Kubernetes using our new helm chart.

See the docs for all the details.

Nomad deployments

Arroyo has long had first-class support for Nomad as a scheduler, where we take advantage of the very low-latency and lightweight scheduling support. Now we also support Nomad as an easy deploy target for the control plane as well via a nomad pack.

See the docs for more details.

  • Support for deploying Arroyo to a nomad cluster by @mwylde in #50

SQL features

With this release we are making big improvements in SQL completeness. Notably, we've made our JSON support much more flexible with the introduction of SQL JSON functions including get_json_objects, get_first_json_object, and extract_json_string.

We've also added support for windowless joins.

Here are some of the highlights:

Connectors, Web UI, and platform support

Arroyo now supports SASL authentication for Kafka and FreeBSD

Fixes

Improvements

The full change-log is available at https://github.com/ArroyoSystems/arroyo/commits/release-0.2.0