Randomizing Very Large Datasets

Consider the problem of randomizing a dataset that is so large, it doesn’t even fit into memory. This article describes how you can do it easily and (relatively) quickly in Python.

These days it is not at all uncommon to find datasets that are measured in Gigabytes, or even Terabytes, in size. That much data can help tremendously in the training process to create robust Machine Learning models. But how can you randomize such large datasets?

article artificial intelligence data data science dataset datasets large datasets machine learning memory python randomization