Dec. 17, 2023, 5:41 p.m. | Sanil Khurana

Towards Data Science - Medium towardsdatascience.com

System Design Series: The Ultimate Guide for Building High-Performance Data Streaming Systems from Scratch!

Source: Unsplash

Setting up an example problem: A Recommendationxt System

“Data Streaming” sounds incredibly complex and “Data Streaming Pipelines” even more so. Before we talk about what that means and burden ourselves with jargon, let’s start with the reason for the existence of any software system, a problem.

Our problem is pretty simple, we have to build a recommendation system for an e-commerce website (something like …

ai building data data science data streaming design example guide jargon kafka performance pipelines programming series software development streaming systems talk

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Senior Machine Learning Engineer (MLOps)

@ Promaton | Remote, Europe

Principal Applied Scientist

@ Microsoft | Redmond, Washington, United States

Data Analyst / Action Officer

@ OASYS, INC. | OASYS, INC., Pratt Avenue Northwest, Huntsville, AL, United States