April 22, 2024, 7 a.m. | Rafal Gancarz

InfoQ - AI, ML & Data Engineering www.infoq.com

Yelp reworked its data streaming architecture by employing Apache Beam and Apache Flink. The company replaced a fragmented set of data pipelines for streaming transactional data into its analytical systems, like Amazon Redshift and in-house data lake, using Apache data streaming projects to create a unified and flexible solution.

By Rafal Gancarz

ai amazon amazon redshift apache apache-beam apache flink apache kafka architecture architecture & design create data data lake data pipelines data streaming event stream processing flink lake ml & data engineering pipelines projects redshift set solution streaming systems the company yelp

Data Engineer

@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania

Artificial Intelligence – Bioinformatic Expert

@ University of Texas Medical Branch | Galveston, TX

Lead Developer (AI)

@ Cere Network | San Francisco, US

Research Engineer

@ Allora Labs | Remote

Ecosystem Manager

@ Allora Labs | Remote

Founding AI Engineer, Agents

@ Occam AI | New York