Oct. 17, 2023, 7:07 p.m. | Jeff Chou

Towards Data Science - Medium towardsdatascience.com

We ran a $12K experiment to test the cost and performance of Serverless warehouses and dbt concurrent threads, and obtained unexpected results.

By: Jeff Chou, Stewart Bryson

Image by Los Muertos Crew

Databricks’ SQL warehouse products are a compelling offering for companies looking to streamline their production SQL queries and warehouses. However, as usage scales up, the cost and performance of these systems become crucial to analyze.

In this blog we take a technical deep dive into the cost and …

apache spark cloud computing companies cost databricks data engineering dbt experiment jeff lessons learned performance production products serverless sql sql queries test testing threads warehouse warehouses

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Senior Machine Learning Engineer (MLOps)

@ Promaton | Remote, Europe

Robotics Technician - 3rd Shift

@ GXO Logistics | Perris, CA, US, 92571