July 24, 2023, 10 a.m. | Rafal Gancarz

InfoQ - AI, ML & Data Engineering www.infoq.com

Grammarly adopted the medallion architecture while migrating from their in-house data lake, storing Parquet files in AWS S3, to the Delta Lake lakehouse. The company created a new event store for over 6000 event types from 40 internal and external clients and, in the process, improved data quality and reduced the data-delivery time by 94%.

By Rafal Gancarz

ai apache spark architecture architecture & design aws aws s3 big data case study data databricks data lake data quality data warehouse delta development etl event event stream processing files grammarly lake lakehouse ml & data engineering parquet platform process quality spark streaming types

More from www.infoq.com / InfoQ - AI, ML & Data Engineering

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Senior Machine Learning Engineer (MLOps)

@ Promaton | Remote, Europe

Robotics Technician - 3rd Shift

@ GXO Logistics | Perris, CA, US, 92571