1.5 Years of Spark Knowledge in 8 Tips

My learnings from Databricks customer engagements

After working with ~15 of the largest retail organizations for the past 18 months, here are the Spark tips I commonly repeat. Throughout this post, we assume a general working knowledge of spark and it’s structure, but this post should be accessible to all levels of spark.

Let’s dive in!

0 — Quick Review

Quickly, let’s review what spark does… …

apache apache spark author customer databricks data engineering data science general image knowledge machine learning organizations python retail spark technical the spark tips