The unglamorous 80%

Real data is messy: missing values, typos, inconsistent formats, duplicates, and outliers. Most of a data scientist's time goes here — and doing it well is what separates trustworthy analysis from garbage-in-garbage-out.

Your cleaning checklist

• Missing values — drop, impute (mean/median/mode), or flag.
• Types & formats — dates as dates, numbers as numbers.
• Duplicates — find and resolve them.
• Outliers — detect, then decide: error or signal?
• Consistency — "USA" vs "U.S." vs "United States".

df = df.drop_duplicates()
df["date"] = pd.to_datetime(df["date"])
df["revenue"] = df["revenue"].fillna(df["revenue"].median())

Golden rule: document every cleaning decision — future-you (and your reviewers) will thank you.