Nov. 9, 2023, 8:54 p.m. | /u/mundus108

Data Science www.reddit.com

I'm taking over a project that will involve receiving 15-20M rows of data monthly, do some basic analysis on them (just sorting/deduping), and then distributing this data to some 3rd party companies.

I have been more on the Data Analyst side of things and while I'm proficient with R, SQL and Python, but I've never had to build a storage/pipeline nor have I worked with this amount of data at once before. It makes sense to use a third party …

advice analysis analyst basic companies data data analyst datascience medical medical data project sorting them

Founding AI Engineer, Agents

@ Occam AI | New York

AI Engineer Intern, Agents

@ Occam AI | US

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne