Detect anomalies in time series with Isolation Forest
This tutorial shows how to detect anomalies in time series with Isolation Forest, explaining why to create temporal features and how to use the technique to flag outliers. It is useful to maintain data quality, monitor systems or alert on rare production events. The approach is lightweight, does not replace manual validation and works well when there are not many anomaly labels available.
Prerequisites
- Python 3.7+ with pandas and scikit-learn installed
- Basic knowledge of Python and time series
- CSV file with two columns: timestamp and value (e.g.: timestamp, value)
Ideally have at least a few hundred points (for example 1k–100k). For very short series (for example <200 points) statistical indicators can be unstable and additional validation is advisable. For hourly series think in windows of 24 or 168; for irregular-frequency series normalize to a fixed frequency before extracting features.
Step 1: Load the data and understand the problem
First load the time series and check frequency and missing values. It is important to know whether the series is daily, hourly, or has a different periodicity because temporal features (lags, windows) depend on that. Also inspect for obvious outliers: a spike with 10x the mean amplitude may be a legitimate anomaly.
import pandas as pd
df = pd.read_csv('serie.csv', parse_dates=['timestamp'])
df = df.sort_values('timestamp').reset_index(drop=True)
print(df.head())
print(df['timestamp'].diff().value_counts().head())
Practical example: if the most common difference is 1H you have an hourly series; if it is 1D, a daily series. If there are large gaps consider resampling and filling with interpolation or using missingness indicators.
Step 2: Prepare features to detect anomalies
Isolation Forest works better with features that express temporal behavior and local variations. Create rolling windows (rolling mean/std), differences (lags) and cyclical features (hour of day, day of week). For hourly series, common windows: 3, 24, 168 (3h, 1 day, 1 week). For daily series, windows: 7, 30, 90.
import numpy as np
# assumir df com 'timestamp' e 'value'
df['value'] = df['value'].astype(float)
# características temporais básicas
df['hour'] = df['timestamp'].dt.hour
df['dayofweek'] = df['timestamp'].dt.dayofweek
# janelas móveis
df['roll_mean_3'] = df['value'].rolling(3, center=False, min_periods=1).mean()
df['roll_std_3'] = df['value'].rolling(3, center=False, min_periods=1).std().fillna(0)
# diferença em relação à janela anterior
df['diff_1'] = df['value'].diff().fillna(0)
features = ['value', 'roll_mean_3', 'roll_std_3', 'diff_1', 'hour', 'dayofweek']
X = df[features].fillna(0)
Note: you can also include specific lags (value.shift(24)), seasonal indicators (value from the same period in the previous week) and cyclical encoding for hour: sin/cos(hour*2pi/24) to avoid discontinuities at the 23→0 transition.
Step 3: Scale and train the Isolation Forest
Scaling helps prevent variables with large amplitude from dominating the decision. Then train the Isolation Forest; tune the contamination parameter to the expected percentage of anomalies. If you expect ~1% anomalies use contamination=0.01; for more sensitive systems, 0.005 or less.
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# contamination: expected proportion of anomalies (e.g.: 0.01 = 1%)
model = IsolationForest(n_estimators=100, contamination=0.01, random_state=42)
model.fit(X_scaled)
# -1 = anomaly, 1 = normal
labels = model.predict(X_scaled)
df['anomaly'] = (labels == -1).astype(int)
# score: more negative values are more anomalous
df['anomaly_score'] = model.decision_function(X_scaled)
Practical tip: n_estimators=100 is a good compromise; increasing to 200–500 can improve stability but increases training time. With 100k points, n_estimators=100 typically trains in a few seconds on a modern machine; test and measure.
Step 4: Tune and reduce false positives
Not everything that is rare is a failure; therefore it is common to tune the model to reduce alerts. Try changing contamination, swapping features or using thresholds on the anomaly_score. Another technique is to require persistence: confirm an anomaly only if 2+ consecutive points occur or if the anomaly exceeds a percentile.
# exemplo simples: exigir 2 pontos consecutivos para confirmar anomalia
df['anomaly_confirmed'] = 0
for i in range(1, len(df)):
if df.loc[i, 'anomaly'] == 1 and df.loc[i-1, 'anomaly'] == 1:
df.loc[i-1:i, 'anomaly_confirmed'] = 1
# alternativa: usar percentil do score para definir corte
threshold = np.percentile(df['anomaly_score'], 1) # 1% mais anómalos
df['anomaly_by_score'] = (df['anomaly_score'] <= threshold).astype(int)
Other strategies: group anomalies close in time (time-based clustering), apply a median filter to reduce noise, or combine with heuristic rules (for example, ignore spikes that coincide with scheduled maintenance).
Verify the result
Confirm that the identified anomalies make sense: inspect timestamps and values and compare with operational logs. If you have true labels compute precision and recall. Without labels, use manual sampling: review 50–200 identified cases and adjust contamination until you reach an acceptable false positive rate (e.g.: <10%).
# mostrar anomalias detetadas
print(df[df['anomaly_confirmed'] == 1][['timestamp','value','anomaly_score']].head())
# se tiver labels verdadeiras (coluna 'is_true_anomaly') pode calcular:
# from sklearn.metrics import precision_score, recall_score
# print(precision_score(df['is_true_anomaly'], df['anomaly_confirmed']))
Conclusion
With Isolation Forest and simple temporal features you can detect anomalies in time series in a practical and efficient way. Try varying windows (e.g.: 24/168), include seasonal lags and adjust contamination according to the expected prevalence. For more sophisticated cases combine with time series models (e.g.: Prophet + residuals) or with human validation to reduce alerts. Final tip: define an operational acceptability KPI (how many false positives per day are tolerable?) and tune the process according to that operational objective.