Python’s standard-library `csv` module reads and writes CSV and other delimited text files. Use it instead of splitting lines by hand: quoted fields can contain delimiters, and the module handles those fields when reading or writing. section 21.2.4 section 21.5.1
Choose an interface to match the records:
- `csv.reader(file, delimiter="|")` reads pipe-delimited records as lists of fields; omit `delimiter` for comma-delimited input. section 21.2.4
- `csv.writer(file).writerows(rows)` writes a sequence of rows. Open the output with `newline=''` as shown in the file-writing example. section 21.5.1
- `csv.DictWriter(file, fieldnames=fields)` writes dictionary rows in the specified field order; call `writeheader()` to add column names. section 21.5.1
Read a quoted field, then write the row
`rows = list(csv.reader(['Illinois,"Jan 01, 1979",17.48']))` produces `[['Illinois', 'Jan 01, 1979', '17.48']]`: the comma inside the quoted date does not start another field. To save the rows, open a destination file with `newline=''` and pass them to `csv.writer(destination).writerows(rows)`. section 21.2.4 section 21.5.1
results2 = [fields for fields in csv.reader(open("temp_data_01.csv", newline=''))]
results2
[['Notes', 'State', 'State Code', 'Month Day, Year', 'Month Day, Year
Code', 'Avg Daily Max Air Temperature (F)', 'Record Count for Daily Max Air
Temp (F)', 'Min Temp for Daily Max Air Temp (F)', 'Max Temp for Daily Max
Air Temp (F)', 'Avg Daily Max Heat Index (F)', 'Record Count for Daily Max
Heat Index (F)', 'Min for Daily Max Heat Index (F)', 'Max for Daily Max
Heat Index (F)', 'Daily Max Heat Index (F) % Coverage'], [], ['',
'Illinois', '17', 'Jan 01, 1979', '1979/01/01', '17.48', '994', '6.00',
'30.50', 'Missing', '0', 'Missing', 'Missing', '0.00%'], ['', 'Illinois',
'17', 'Jan 02, 1979', '1979/01/02', '4.64', '994', '-6.40', '15.80',
'Missing', '0', 'Missing', 'Missing', '0.00%'], ['', 'Illinois', '17', 'Jan
03, 1979', '1979/01/03', '11.05', '994', '-0.70', '24.70', 'Missing', '0',
'Missing', 'Missing', '0.00%'], ['', 'Illinois', '17', 'Jan 04, 1979',
'1979/01/04', '9.51', '994', '0.20', '27.60', 'Missing', '0', 'Missing',
'Missing', '0.00%'], ['', 'Illinois', '17', 'May 15, 1979', '1979/05/15',
'68.42', '994', '61.00', '75.10', 'Missing', '0', 'Missing', 'Missing',
'0.00%'], ['', 'Illinois', '17', 'May 16, 1979', '1979/05/16', '70.29',
'994', '63.40', '73.50', 'Missing', '0', 'Missing', 'Missing', '0.00%'],
['', 'Illinois', '17', 'May 17, 1979', '1979/05/17', '75.34', '994',
'64.00', '80.50', '82.60', '2', '82.40', '82.80', '0.20%'], ['',
'Illinois', '17', 'May 18, 1979', '1979/05/18', '79.13', '994', '75.50',
'82.10', '81.42', '349', '80.20', '83.40', '35.11%'], ['', 'Illinois',
'17', 'May 19, 1979', '1979/05/19', '74.94', '994', '66.90', '83.10',
\'82.87', '78', '81.60', '85.20', '7.85%']]
The parsed temperature data shows quoted, comma-containing fields kept intact. It also shows an empty list for a blank input row, which downstream processing may need to handle. section 21.2.4
Parsing is only one step in a data-cleaning job. For the Illinois weather data, filter the desired county, stop before the trailing documentation, replace `Missing`, convert percentages, and then write the selected records. Check field positions carefully: searching an entire row for `Cook` is not the same as checking the county field. section 21.6 section 21.6.2
