Feb. 7, 2022, 10:23 p.m. | Oleksandr

DEV Community dev.to




Table Of Contents


  1. Synopsis

  2. Rationale

  3. Context

  4. The Recipe

  5. Caveats





Synopsis

This article provides a recipe for building fast look-up tables for huge CSV data sets in Python.


Look-up tables are meant to be read-only data structures used to search for a presence of a given key and to retrieve multiple values associated with existing keys.


Recipe is meant to be a rough approach in contrast to a precise algorithm.


Huge is relative and means significantly exceeding the amount of available …

bigdata csv data numpy python ups

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Senior Machine Learning Engineer (MLOps)

@ Promaton | Remote, Europe

Staff Software Engineer, Generative AI, Google Cloud AI

@ Google | Mountain View, CA, USA; Sunnyvale, CA, USA

Expert Data Sciences

@ Gainwell Technologies | Any city, CO, US, 99999