A regular expression is a sequence of characters that specifies a text pattern. It searches, matches, and extracts values from structured or unstructured text so the results can be parsed into usable data. section 10.2.2
Use regular expressions in a data-cleaning workflow to:
- Identify numeric patterns while excluding surrounding characters. section 10.1.1
- Extract temperature integers from strings such as Fahrenheit readings. section 10.2.2
- Capture wind-speed values, including values followed by the unit “mph,” before converting them to integers. section 10.2.2
- Prepare cleaned columns for grouping, aggregation, and regression-model input. section 10.1.1
Extracting weather values
A forecast field may contain text rather than a number. Applying re.findall(r'\d+', row) retrieves the integers from a wind-speed string, while re.search(r'\d+', t).group() retrieves the temperature integer. Matching r"\d+ mph" captures wind-speed values that include their unit; the extracted values can then be converted and aggregated by date. section 10.1.1 section 10.2.2
import re #1
def get_wind_speed(row): #2
numbers = [int(num) for num in re.findall(r'\d+', row)]
return max(numbers)
forecast_df["date"] = pd.to_datetime(forecast_df['endTime']).dt.date #3
forecast_df["wind_speed"] = forecast_df["windSpeed"].apply(
get_wind_speed
) #4
forecast_df[["date", "temperature", "wind_speed"]].head() #5
Inspect a complete Python example that imports re, extracts integers from wind-speed text, and adds cleaned date and wind-speed columns to a pandas dataframe. section 10.1.1
