Anomaly detection (fraud)

Flag anomalous transactions with AnomalyDetector. It wraps PyOD with two modes: a hands-off auto engine that serves the z-score mean of several detectors, and a single named detector.

Install extras

PyOD (the classical detectors — ecod, iforest, hbos, cblof, …) ships in the ml4 core, so anomaly detection works out of the box. parleml-ml4[unsupervised] only adds UMAP (2-D projection) and neural text embeddings. Deep detectors (autoencoder, vae, dif) additionally need pyod[torch]; suod needs pyod[suod].

1. A frame with planted outliers

import numpy as np
import pandas as pd

rng = np.random.default_rng(0)
n = 2_000
amount = rng.normal(60, 15, n).clip(1, None)
hour = rng.integers(0, 24, n)

# Plant ~2% obvious fraud: huge amounts at odd hours.
idx = rng.choice(n, size=int(0.02 * n), replace=False)
amount[idx] = rng.normal(2_000, 400, idx.size)
hour[idx] = rng.integers(1, 5, idx.size)

df = pd.DataFrame({"amount": amount.round(2), "hour": hour,
                   "channel": rng.choice(["web", "pos", "atm"], n)})

2. Auto mode — the z-score mean of several detectors

mode="auto" profiles the data, picks detectors from PyOD's benchmark-backed knowledge base, runs them, and serves the mean of their standardised scores, beside ADEngine's rank consensus and a quality verdict.

On tabular data the candidates come from a routing table of 8 names, and at most 3 are run, sequentially — so auto reaches 8 of the 23 detectors in the single-mode roster. Reach the rest with mode="single". priority only bites at n_features >= 100 and only for "speed" / "accuracy"; the default "balanced" matches no rule at all.

from parleml_ml4.unsupervised import AnomalyDetector
from parleml_ml4.config.anomaly import AnomalyConfig

det = AnomalyDetector(AnomalyConfig(mode="auto", priority="balanced")).fit(df)
result = det.result_

print(result.n_anomalies, "of", result.n_samples, "rows flagged")
print(det.explain())              # which detectors ran + the quality report

contamination applies in auto mode too

ml4 writes the configured value into each detector plan before the engine runs, and ADEngine respects a caller-supplied value — it fills its own knowledge-base default (0.1) only when nobody supplies one.

result.contamination still reports the realised flagged rate rather than echoing the request back: in auto the labels come from the z-score mean of every candidate (detector_name="zscore_mean"), whose ties can move the achieved rate off the requested one.

3. Or a single named detector

Pick one estimator from the 23-detector roster and control its knobs directly:

cfg = AnomalyConfig(mode="single", detector_name="iforest",
                    contamination=0.02, detector_kwargs={"n_estimators": 200})
det = AnomalyDetector(cfg).fit(df)

4. Read scores and score new data

labels = det.result_.labels          # (n,) in {0, 1}
scores = det.result_.scores          # higher = more anomalous
top    = det.result_.top_k_indices   # most-anomalous rows first

new_labels = det.predict(df.head())          # {0, 1} for new rows
new_scores = det.decision_function(df.head())
reasons    = det.explain_rows(df.head())     # per-row top contributing features

det.contamination_diagnostics() returns a threshold-sensitivity curve, in the same raw score units as decision_function, so the curve and the run's own labels agree.

If you have ground-truth labels, score them yourself with sklearn.metrics.roc_auc_score / average_precision_score against det.decision_function(X). ml4 does not ship a validate() helper: the metric would be computed in-sample on the rows the detector was fitted on, which is not an estimate of anything, and a one-line sklearn call makes the choice of data explicit.

See the Unsupervised guide and the AnomalyDetector reference.

Next: Clustering · Unsupervised guide