Anomaly detection identifies behavior that is valid but unusual relative to what a system normally does. In a digital twin, it turns continuous sensor readings into early warnings of process drift, equipment degradation, or safety concerns—even when no predefined fault signature matches. section 8.2 section 1.2.3
Three related capabilities answer different questions:
- Fault detection asks whether a known abnormal condition has occurred, such as a sensor stopping or reporting outside its operational range. section 8.2
- Anomaly detection asks whether an otherwise valid pattern differs from expected behavior. section 8.2
- Diagnostics asks what likely caused a detected fault or anomaly. section 8.2
A simple statistical method is the z-score: subtract the historical mean from the current measurement, then divide by the standard deviation. A large absolute score marks a reading far from its baseline. Rolling statistics make that baseline local rather than imposing one fixed threshold on every situation. Such methods are fast and suited to a single measured variable, but they rely on assumptions about the data’s distribution. section 8.2.2
Indoor air-quality readings
A fixed PM2.5 rule that flags every reading above 10 μg/m³ would flag 15 μg/m³ even when a nearby bushfire has raised the day’s baseline to 12 μg/m³. A rolling z-score instead tests whether the reading is unusual in its current context. In one implementation, pandas reads a Parquet feature matrix, drops missing PM2.5 values, calculates rolling mean and standard deviation, and selects readings whose absolute z-score exceeds 3. The code uses a 24-row window; calling it 24 hours depends on the readings’ cadence. section 8.2.2
Listing 8.4 Calculating z-score for indoor PM2.5 levels in my home
import pandas as pd
import numpy as np
df = pd.read_parquet("../datapipeline/2025_export.parquet")
target_col = "pm25_living_room"
analysis_df = df[[target_col]].dropna().copy()
analysis_df.rename(columns={target_col: "value"}, inplace=True)
window_size = 24 #1
analysis_df["rolling_mean"] = analysis_df["value"].rolling( #2
window=window_size).mean()
analysis_df["rolling_std"] = analysis_df["value"].rolling( #3
window=window_size).std()
analysis_df["z_score"] = (
(analysis_df["value"] - analysis_df["rolling_mean"]) /
analysis_df["rolling_std"]
).bfill()
november_df = analysis_df.loc["2025-11"].copy()
z_threshold = 3 #4
anomalies = november_df[np.abs(november_df["z_score"]) > z_threshold]
print(f"Air Quality Anomalies detected in November: {len(anomalies)}")
The listing shows the Parquet input, rolling calculations, and three-standard-deviation selection in pandas. section 8.2.2
A rolling baseline changes as new readings arrive. Once a spike enters the window, it can raise both the mean and standard deviation, so later spikes may no longer trigger an alert. This can reduce repetitive alerts and alert fatigue, but it also means a flagged event should not be mistaken for a diagnosis. Charts and sparklines offer a complementary way to inspect unusual sensor patterns; forecasting is a separate step toward predicting future behavior. section 8.2.2 section 8.2 section 6.2.1 section 8.3
