Data Cleaning: Automatically Removing Bad Data

Purging wrong data-type entries from numeric and character columns

Cleaning data is almost always one of the first steps you need to take after importing your dataset. Pandas has lots of great functions for cleaning, with functions like isnull(), dropna(), drop_duplicates(), and many more. However, there’s two major situations that aren’t covered:

• A would-be numeric column is littered with strings and booleans
• An aspiring character column has sporadic numeric values and booleans

This bad …

bad-data data data cleaning data-type pandas pandas-dataframe