AnomalyResult & ClusteringResult¶

What a fitted detector puts on result_. Most fields are self-explanatory; the ones below exist to stop a number being read as something it is not.

In short

Clustering quality indices (*_features) are measured in feature space; metric_coverage says how many rows they saw.
Check verdict before any score: it says whether the run found clusters at all.
The anomaly headline is separation, not anomaly_ratio.
Read distinctive fields with distinctive_fields(profiles, cluster_id). The table is descriptive and carries no p-value.
Every negative cluster label is noise; cluster_profiles keeps those rows, so filter to cluster_id >= 0 for one row per segment.

Reading the numbers¶

Clustering quality is measured in feature space. silhouette_features, davies_bouldin_features and calinski_harabasz_features are computed on the un-reduced fused matrix — the space your data is actually in. The _features suffix is deliberate: the auto-k sweep scores in the (possibly UMAP-reduced) clustering geometry, and geometry_space names that space. Three caveats apply to the reported index, and the result carries the evidence for each:

Question	Field
How much of the data did the metric see?	`metric_coverage` — noise rows are excluded
Was it a subsample?	Compare `silhouette_sample_size` (rows the metric consumed — the scored rows after the noise mask, and after the draw where the cap bit) against `n_samples - n_noise`: equal ⇒ every eligible row, smaller ⇒ a draw. `None` ⇒ nothing was scored
How much of the score is search?	`auto_k_selected_score` vs `auto_k_median_score`

Did the run find anything at all? verdict is clustered, single_cluster, all_noise or degenerate, with verdict_reason as a human sentence. A quality score answers "how separated", not "is there structure" — those are different questions. verdict keeps a failed run distinct from a merely undefined metric.

The anomaly headline is separation, not anomaly_ratio. anomaly_ratio is the flagged fraction, which in single mode equals the contamination setting by construction — it ranks runs by a knob the user set. separation is (median(score | flagged) − median(score | unflagged)) / IQR(score | unflagged): dimensionless, shift- and scale-invariant, ≈0 when nothing was found, and large on real outliers at any contamination. It is None for the degenerate cases rather than a fabricated number.

Do not confuse it with quality["engine_separation_ratio"] (auto mode). That is ADEngine's saturating ratio of mean consensus scores, one third of a heuristic verdict. quality carries no separation key at all, so the two cannot collide — including inside quality["explanation"], which is ml4's own one-line restatement of the terms rather than PyOD's format string over the raw ones. A term this run could not measure prints as n/a there, matching the None on the key itself.

consensus_scores is always in (0, 1]. When only one planned detector succeeded, PyOD returns that detector's raw decision_scores_ instead of a rank-normalised mean, so ml4 rank-normalises it before storing; quality["consensus_basis"] names which path produced it, and every per_detector_results[i]["consensus_spearman"] is None on that path rather than a self-correlation of 1.0.

contamination means different things per mode — the requested rate in single, the achieved rate in auto — so read it as "what happened".

score_degeneracy is set when the scores are not a usable ranking (all equal, or numerical noise about zero). When present, the top-K table and the findings are not measurements and must not be rendered as such.

caveats is a list of plain-language notes about the run — always present, often empty.

Noise rows and the per-cluster surfaces¶

Density algorithms label rejected rows negative: -1 for noise, and sklearn's HDBSCAN additionally emits -2 for rows containing infinities and -3 for rows with missing data. Everything that counts or scores clusters treats every negative label as noise.

The per-cluster surfaces are deliberately not uniform about this:

Surface	Negative labels
`cluster_keywords`, `cluster_keywords_by_column`, `passive_field_stats`	excluded
`n_clusters`, `n_noise`, the validity indices	excluded (and `metric_coverage` reports how much was left)
`cluster_profiles`	included

cluster_profiles keeps them on purpose — "what do the rejected rows look like?" is a fair question, and the profile is the only surface that can answer it. The contract is therefore on the consumer: anything rendering one row per discovered segment must filter to cluster_id >= 0 first. parleml_ml4.unsupervised.labels exposes non_noise_mask / cluster_ids so that rule lives in one place rather than being re-derived per caller.

Distinctive fields: the effect size and the decision¶

cluster_profiles carries two different numbers per (cluster, axis) and they answer different questions. Conflating them gives a three-row cluster a confident, model-written business name derived from noise.

Column	Question it answers
`z_vs_global`	How far apart are they? The cluster mean minus the global mean over the feature's population sd. Deliberately carries no sample-size term, so it stays a comparable effect size.
`separation_score`	How does this source rank against the others? A ranking key only — axis-count-calibrated by Šidák so a wide source cannot win on width, constant across a source's axes. Not a probability; do not render it as one.
`source_representative`	May this field be reported? `True` on exactly one axis per (cluster, source) — the source's strongest — and only when the cluster clears the support floor. One axis per source is what stops a 384-dim text column filling a "top 8" card by itself.

There is no p-value here, and there cannot be

The profile carries no p_value, source_p_value, source_q_value or distinctive column, and ClusteringConfig has no FDR knob. Any such p-value would test the null "this cluster is a random subset of the rows" — which is false by construction: the labels were produced by an algorithm maximising exactly this separation, on exactly this matrix. That is post-selection inference, and its certificate is void. Measured on structure-free data, such a test fires on hundreds of fields in every fitted KMeans cluster, against 0 under a genuinely random labelling. A false-discovery rate that reports the opposite of the truth is worse than no rate at all. This table is descriptive: these are the fields that most distinguish the cluster, and nothing here says they could not have arisen by chance.

Read the ranked answer with parleml_ml4.unsupervised.clustering.reporting.distinctive_fields(profiles, cluster_id) — do not re-derive it from max(|z_vs_global|). Every surface that shows "distinctive fields" (the card, the σ badge, the LLM cluster namer's evidence) must render that one list, or it re-opens both of:

No support floor. A three-row cluster's per-feature z has null sd 1/sqrt(3) = 0.58, so roughly two thirds of its features clear a |z| > 0.25 bar on noise alone. The floor is parleml_ml4.unsupervised.passive_stats.MIN_CLUSTER_SUPPORT (10 rows) — the same constant the passive-field surface uses, imported rather than re-declared, so two cards on one page cannot disagree about whether a small cluster may claim a distinctive field.
A max over k axes is not axis-count-fair. Grouping engineered axes back to their source column takes the strongest of the source's axes, and E[max_{i<=k}|z_i|] grows like sqrt(2 ln k) — ~2.4 null sd's for a 128-dim embedding. Un-calibrated, a wide text column out-ranks a genuinely shifted numeric column purely by being wide. The Šidák step prices k in; a wide source with real signal still wins.

An empty result is a real answer — "no field distinguishes this cluster at this confidence" — and consumers must say so rather than fall back to an uncorrected ranking.

What `z_vs_global` is a z of¶

A z-score is invariant under an affine per-column transform, so on those axes the z on the preprocessed matrix really is the z of the raw column. That is not true of every axis the preprocessor builds, and the z_units column says which case each one is:

`z_units`	Axis	Safe wording
`raw_affine`	1:1 numeric/datetime under `numeric_scaler` `robust` or `standard`	"standard deviations above the dataset average"
`rank_normal`	1:1 numeric under `numeric_scaler="quantile"` (a monotone rank transform, not affine)	"relative enrichment"
`share`	a 0/1 indicator — one-hot level, `__missing` / `__rare` / `__unseen` — standardised by the Bernoulli sd and then bounded by `indicator_z`	"over-/under-represented"
`frequency`	a high-cardinality `__freq` axis (a lookup, affine in nothing)	"relative enrichment"
`latent`	a text latent axis matched by name — `_TSVD_` or `__emb_*` — there is no raw column for it to be the z of	"relative enrichment", and the sign is not meaningful (`magnitude` is `True`)
`engineered`	any other axis derived from a source column by a transform the classifier cannot name — including the NMF topic axes, which the text preprocessor names `<col>_WT_i`, so they land here rather than in `latent`	"relative enrichment", exactly as `latent`; the two differ only in whether the transform could be named
`unknown`	no source map was available (the caller supplied its own matrix with `preprocessor=None`)	"relative enrichment"

AnomalyResult¶

AnomalyResult `dataclass` ¶

AnomalyResult(
    labels: ndarray,
    scores: ndarray,
    decision_threshold: float,
    detector_name: str,
    mode: Literal["auto", "single"],
    consensus_scores: Optional[ndarray] = None,
    quality: Optional[dict[str, Any]] = None,
    per_detector_results: Optional[
        list[dict[str, Any]]
    ] = None,
    per_detector_labels: Optional[ndarray] = None,
    per_detector_names: Optional[list[str]] = None,
    scoring_rule: Optional[str] = None,
    caveats: list[str] = list(),
    score_degeneracy: Optional[dict[str, Any]] = None,
    findings: Optional[list[dict[str, Any]]] = None,
    keywords_by_column: Optional[
        dict[str, dict[int, list[tuple[str, float]]]]
    ] = None,
    n_samples: int = 0,
    n_features_in: int = 0,
    feature_names_in: list[str] = list(),
    runtime_seconds: float = 0.0,
    contamination: float = 0.05,
    score_summary: dict[int, float] = dict(),
    top_k_indices: ndarray = (
        lambda: empty(0, dtype=int64)
    )(),
    top_k_scores: ndarray = (
        lambda: empty(0, dtype=float64)
    )(),
)

Frozen container produced by :class:AnomalyDetector.

For mode='auto' (ADEngine) the rank-consensus / quality / per-detector / scoring-rule fields are populated. For mode='single' they are None.

labels is the in-sample labelling — for LOF / KNN it legitimately disagrees with AnomalyDetector.predict() on the very rows that were fitted. That is PyOD / scikit-learn semantics, not a defect: the two answer different questions.

labels / scores are PyOD's labels_ / decision_scores_ — "which of the rows I was fitted on are anomalous?" For LOF that is the standard LOF (negative_outlier_factor_).
AnomalyDetector.predict(X) is PyOD's predict — "are these rows anomalous novelties relative to what I was fitted on?" For LOF that is score_samples, whose scikit-learn docstring warns that a training point handed back in "considers the later in its own neighborhood": it becomes its own nearest neighbour, looks locally dense, and so reads as an inlier.

Both sides apply the same decision_threshold; only the score source differs, and only for the neighbour-based detectors. Measured across the whole dispatch table on a 220-row frame, lof (max abs diff 0.46) and knn (0.28) are the only two that diverge materially; every other curated detector re-derives the same scores bitwise or to floating-point noise (≤1e-13), so its labels match. Concretely, on that frame — 20 planted outliers — lof reports 16 here while predict flags 2 on those same rows. On genuinely unseen rows LOF / KNN rightly decline to flag a repeat of that outlier cluster — it was in the training set, so it is a known dense region rather than a novelty. Report labels for the fitted run; use predict to score new data. set_operating_threshold does not bridge the two (it also routes through decision_function). Full evidence: docs/internal/ml4/_false_positives.md.

anomaly_ratio `property` ¶

anomaly_ratio: float

Share of rows flagged in-sample.

In mode='single' this is a restatement of the request, not a measurement. PyOD thresholds at the 1 - contamination quantile, so the flagged share equals config.contamination up to ties — which is why it is reported beside :attr:~AnomalyResult.contamination and its target / achieved reading, and why :attr:separation rather than this is the run's quality score.

separation `property` ¶

separation: Optional[float]

Quality of this run's ranking, independent of where it was cut.

:func:separation_statistic over :attr:scores and :attr:labels — see it for the definition and for why anomaly_ratio cannot serve as a headline score. None whenever the number would be fabricated (nothing flagged, everything flagged, a zero unflagged IQR).

Not to be confused with quality['engine_separation_ratio'], which is ADEngine's saturating ratio of mean rank-consensus scores.

A property, not a stored field, so it cannot drift from the scores / labels it summarises: there is no second copy to fall out of step when a consumer re-thresholds.

Returns None outright when :attr:score_degeneracy is set: those scores are already diagnosed as noise, and a separation computed from a labelling that was decided by rounding would be a confident number on top of a refusal. That field is the single signal for "the scores carry no information"; this one does not invent a second.

ClusteringResult¶

ClusteringResult `dataclass` ¶

ClusteringResult(
    labels: ndarray,
    soft_proba: Optional[ndarray] = None,
    algorithm: str = "",
    n_clusters: int = 0,
    n_noise: int = 0,
    cluster_sizes: dict[int, int] = dict(),
    verdict: Literal[
        "clustered",
        "single_cluster",
        "all_noise",
        "degenerate",
    ] = "clustered",
    verdict_reason: str = "",
    silhouette_features: Optional[float] = None,
    davies_bouldin_features: Optional[float] = None,
    calinski_harabasz_features: Optional[float] = None,
    silhouette_sample_size: Optional[int] = None,
    metric_coverage: float = 1.0,
    geometry_space: str = "features",
    dbscan_eps_used: Optional[float] = None,
    birch_threshold_used: Optional[float] = None,
    n_samples: int = 0,
    n_features_in: int = 0,
    feature_names_in: list[str] = list(),
    runtime_seconds: float = 0.0,
    cluster_profiles: Optional[DataFrame] = None,
    cluster_keywords: Optional[dict[int, list[str]]] = None,
    cluster_keywords_by_column: Optional[
        dict[str, dict[int, list[tuple[str, float]]]]
    ] = None,
    cluster_representatives: Optional[
        dict[int, list[str]]
    ] = None,
    passive_field_stats: Optional[
        dict[int, dict[str, Any]]
    ] = None,
    auto_k_sweep: Optional[dict[int, float]] = None,
    auto_k_score_name: Optional[str] = None,
    auto_k_n_candidates: Optional[int] = None,
    auto_k_selected_score: Optional[float] = None,
    auto_k_median_score: Optional[float] = None,
    auto_k_skipped_k: Optional[list[int]] = None,
    caveats: list[str] = list(),
)

Frozen container produced by :class:ClusteringDetector.

Two spaces, named. A clustering run can involve two different feature matrices: the fused, un-reduced features (real column names, what the user thinks of as their data) and the clustering geometry the estimator actually optimised in, which is the same matrix unless a pre-reduction ran. Every number here that depends on the choice says which one it means:

silhouette_features / davies_bouldin_features / calinski_harabasz_features and cluster_profiles — fused features, always, whatever the geometry was.
auto_k_sweep / auto_k_selected_score / auto_k_median_score — the geometry, because the sweep must judge candidates in the space the estimator is optimising. geometry_space names it.

They used to share the bare name silhouette while being computed in different spaces, so the platform's headline clustering score was a UMAP embedding's own separation — a quantity UMAP's loss maximises by construction — and the UI showed a second, contradictory silhouette one tab away.

Next: UnsupervisedPipeline · Unsupervised guide

AnomalyResult & ClusteringResult¶

Reading the numbers¶

Noise rows and the per-cluster surfaces¶

Distinctive fields: the effect size and the decision¶

What z_vs_global is a z of¶

AnomalyResult¶

AnomalyResult dataclass ¶

anomaly_ratio property ¶

separation property ¶