A trend is a systematic directional change across an ordered sequence, especially over time. In anomaly detection, normal individual readings can still form a suspicious upward trend, increasing volatility, or another unusual temporal pattern. section 12.7 section 17.2.3
Represent trends at a useful temporal resolution, then compare entities or account for broader time-series structure.
- Aggregate each entity by minute, hour, day, week, or another period; the chosen resolution can change the apparent pattern. section 12.7
- Engineer simple features such as first and last values, their difference, or the ratio of the mean of the final three observations to the initial three. section 12.7
- Interpret apparent trends alongside cycles and seasonality, because expected recurring patterns can change whether a value is anomalous. section 17.2.3
- Use decomposition to separate long-term trend, seasonal pattern, and residual values before judging unusual observations. section 17.2.3
Suspicious purchasing escalation
Group purchases by staff member, product, and day of the month, then plot count, unit cost, and total cost over dates. For staff 10, product 5, and day 28, counts show a clear upward trend, while unit cost does not; total cost rises with apparently increasing variability. Compare this segment with others before treating it as meaningful. section 12.7
Listing 17.4 Using statsmodels for time-series decomposition
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.tsa.seasonal import seasonal_decompose
np.random.seed(0)
n_points = 2000
dates = np.array('2005-01-01', dtype=np.datetime64) + np.arange(n_points)
data = 12*np.sin(4 * np.pi * np.arange(n_points)/365) + \
50.0*np.array(range(n_points))/n_points + \
np.random.normal(12, 2, n_points)
df = pd.DataFrame({'Values': data}, index=dates)
result = seasonal_decompose(df, model='additive', period=365)
result.plot()
plt.show()
Inspect the decomposition workflow separating original observations, trend, seasonal pattern, and residuals. section 17.2.3
