July 24, 2023, 10 a.m. | Rafal Gancarz

InfoQ - AI, ML & Data Engineering www.infoq.com

Grammarly adopted the medallion architecture while migrating from their in-house data lake, storing Parquet files in AWS S3, to the Delta Lake lakehouse. The company created a new event store for over 6000 event types from 40 internal and external clients and, in the process, improved data quality and reduced the data-delivery time by 94%.

By Rafal Gancarz

ai apache spark architecture architecture & design aws aws s3 big data case study data databricks data lake data quality data warehouse delta development etl event event stream processing files grammarly lake lakehouse ml & data engineering parquet platform process quality spark streaming types

More from www.infoq.com / InfoQ - AI, ML & Data Engineering

Software Engineer for AI Training Data (School Specific)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Python)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Tier 2)

@ G2i Inc | Remote

Data Engineer

@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania

Artificial Intelligence – Bioinformatic Expert

@ University of Texas Medical Branch | Galveston, TX

Lead Developer (AI)

@ Cere Network | San Francisco, US