How to monitor Data Lake health status at scale

Companion GitHub repository: Hands-on Great Expectations with Spark

Build a Data Quality workflow with Great Expectations and Spark

Introduction

At Mediaset, the Data Lake is a fundamental tool used daily by everyone who wants to get some company insights or activate data.

By definition, a Data Lake is “a centralized repository to store all your structured and unstructured data at any scale. You can store data natively from the source without having …

aws data data lake data quality data science great expectations health lake scale