Seamless Data Analytics Workflow: From Dockerized JupyterLab and MinIO to Insights with Spark SQL

This tutorial guides you through an analytics use case, analyzing semi-structured data with Spark SQL. We’ll start with the data engineering process, pulling data from an API and finally loading the transformed data into a data lake (represented by MinIO). Plus, we'll utilise Docker to introduce a best practice for setting up the environment. So, let’s dive in and see how it’s all done!

Table of contents

1. Understanding the building blocks
2. Setting up …

apache spark data analysis data engineering docker sql