Skip to content

Pandas & Plotting

Reading Messy CSVs

8 min+75 XPStep 1/10

read_csv is a negotiation

The dream: df = pd.read_csv('data.csv') just works. The reality: real files have comment lines, footers, European decimals, and 'N/A' sprinkled through number columns. read_csv has an argument for every crime:

  • na_values=['N/A', '?', ''] — declare which strings mean missing
  • parse_dates=['date'] — convert text into real dates
  • dtype={'zip': str} — force a column's type (ZIP codes are labels, not quantities)
  • skiprows=3, skipfooter=2 — hop over headers and footers
  • usecols=['date', 'price'] — load only what you need

When in doubt, read a little first: pd.read_csv('data.csv', nrows=5) — inspect, adjust, then load the whole thing.