Sept. 10, 2023, 5:09 p.m. | Jeff Chou

Towards Data Science - Medium towardsdatascience.com

Realities of company and cloud complexities require new levels of control and autonomy to meet business goals at scale

Image by Cosmin Paduraru

As data teams scale up on the cloud, data platform teams need to ensure the workloads they are responsible for are meeting business objectives. At scale with dozens of data engineers building hundreds of production jobs, controlling their performance at scale is untenable for a myriad of reasons from technical to human.

The missing link today is …

autonomy business cloud complexities control data databricks data engineering data engineers data pipelines data platform data science data teams engineers feedback loop machine learning pipelines platform scale snowflake workloads

AI Engineer Intern, Agents

@ Occam AI | US

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Data Engineer - Takealot Group (Takealot.com | Superbalist.com | Mr D Food)

@ takealot.com | Cape Town