all AI news
Grammarly Replaces Its In-House Data Lake With Databricks Platform Using Medallion Architecture
InfoQ - AI, ML & Data Engineering www.infoq.com
Grammarly adopted the medallion architecture while migrating from their in-house data lake, storing Parquet files in AWS S3, to the Delta Lake lakehouse. The company created a new event store for over 6000 event types from 40 internal and external clients and, in the process, improved data quality and reduced the data-delivery time by 94%.
By Rafal Gancarzai apache spark architecture architecture & design aws aws s3 big data case study data databricks data lake data quality data warehouse delta development etl event event stream processing files grammarly lake lakehouse ml & data engineering parquet platform process quality spark streaming types