Residual error is the portion of a time series left after subtracting modeled trend and seasonality. It represents the series with these larger patterns removed. section 17.2.3
A practical residual-based outlier workflow is:
- Identify the trend and seasonal period, then subtract both components from the observations. section 17.2.3
- Inspect the residuals for unusual values, while also considering anomalies in the trend or seasonal component. section 17.2.3
- Use seasonal_decompose() with an explicit period; statsmodels calculates trend, seasonality, and residuals, using LOESS smoothing for the trend. section 17.2.3
Removing a regular spike
A series with spikes every 10 seconds should contain a spike at 120 seconds. Subtracting the expected seasonal spike can show that the observation is not otherwise anomalous. section 17.2.3
Listing 17.4 Using statsmodels for time-series decomposition
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.tsa.seasonal import seasonal_decompose
np.random.seed(0)
n_points = 2000
dates = np.array('2005-01-01', dtype=np.datetime64) + np.arange(n_points)
data = 12*np.sin(4 * np.pi * np.arange(n_points)/365) + \
50.0*np.array(range(n_points))/n_points + \
np.random.normal(12, 2, n_points)
df = pd.DataFrame({'Values': data}, index=dates)
result = seasonal_decompose(df, model='additive', period=365)
result.plot()
plt.show()
A concise Python decomposition example combines a synthetic sine cycle, an upward trend, noise, and a 365-day period. section 17.2.3
