Transfer SQL-> analytics 30x faster with ConnectorX + arrow + dlt

dlt is a recently released python library for data extraction and loading, the EL in ETL. At dltHub we are big fans of optimising things and integrating those optimisations into our toolkit to enable others to re-use them.

Speed boosts and schema from arrow, dlt for loading with schema evolution

In this example, we combine ConnectorX + Arrow + dlt to extract data and load it to a strongly typed environment 30x faster than classic data transfer via sqlalchemy.

Result: …

analytics arrow big data dataengineering data extraction dlt etl extraction fans faster library loading python schema speed sql them toolkit transfer