DATA & AUTOMATION 3 minute read
A CSV row is more than a comma.
Why quoted fields, empty values and a preview matter when you clean a spreadsheet export.
By Mbiopeter
Notice what a field can contain
A quoted CSV field can contain a comma or a line break. An embedded double quote is represented by two double quotes. Splitting every line on commas loses that structure. The small example below has three fields, even though the quoted company name contains a comma.
name,company,city
Ada,"Research, Design & Code",NairobiMake cleaning a deliberate choice
Trimming spaces and removing duplicate rows may be helpful, but they change data. First identify the columns, inspect a few rows, and decide what the output should mean. A blank field can be meaningful; avoid replacing every missing value with a guess.
- Keep a copy of the original input.
- Choose the cleanup operations you need.
- Compare row counts and a few representative records.
- Preview the result before downloading it.
Try the complete workflow
Data Tidy lets you paste a sample or open a CSV, choose cleanup options, inspect the output and download it. Processing happens in your browser. Its case study explains the implementation and the boundaries of this small demonstration.