Skip to content

Decision support (per-case)

A saved ProfilingResult doubles as a Decision Support payload generator: bind a model with result.justifier(...) and feed in a single raw row to get back a structured CaseExplanation describing what the model did and why — the matched archetype, top-K SHAP drivers, every rule that fired, the nearest historical neighbours, and (optionally) local sensitivity curves. This is the second half of the "profile once, use twice" contract: the same result you use to display the global picture also justifies predictions on new data.

In short

  • Bind once with result.justifier(pipeline=...), then call .justify(row) per row. The pipeline is optional: without it, the model stored in the result scores.
  • Each CaseExplanation holds the prediction, the top SHAP drivers, the matched archetype, the strongest fired rules (5 by default) and the nearest training rows.
  • justify_batch produces the same drivers and rules for a whole frame in one pass, without archetypes or neighbours.
  • ICE, LOCO and an archetype counterfactual are off by default; turn them on per call.

justifier / justify

Bind the model once, then justify rows. Reuse one Justifier for batch / repeated scoring — the inference adapter and per-class view are resolved at bind time, and the nearest-neighbour index stays cached on the result:

justifier = result.justifier(pipeline=loaded_ml4_pipeline)  # optional pipeline-mode
ce = justifier.justify(
    x_raw=row_df,        # single-row DataFrame
    calibrated=True,     # honoured only when a calibrator exists
    top_k_shap=10,
    top_k_neighbors=5,
    include_shap=True,
    include_rules=True,
    include_neighbors=True,
)

For a single ad-hoc row, the one-shot result.justify(row_df, ...) is equivalent to result.justifier(pipeline).justify(row_df, ...):

ce = result.justify(
    x_raw=row_df,
    pipeline=loaded_ml4_pipeline,  # optional — pipeline-mode (raw → model input)
    calibrated=True,
)

The returned CaseExplanation carries:

Field Meaning
prediction The model's actual class label (binary / multiclass), a point value (regression), or None when unlabellable; never a class-column index
decision_threshold the binary cutoff matching the probability flavour
shap_drivers list[ShapDriver] — top contributing features
matched_archetype int | None (None when the row is novel)
fired_rules list[FiredRule] — the strongest mined rules that fired, capped at top_k_rules and ranked by rule_sort (default confidence). Each carries the rule's Train numbers (confidence, support, coverage, lift), measured on the rows it was mined on, and — when the rules were measured on held-out rows — its Test side: precision_holdout, precision_lower_holdout (the lower bound of its Wilson interval) and holds_up. See Train and Test numbers
n_fired_rules_total how many rules fired before the top_k_rules cap (drives a "top N of M" viewer label)
nearest_indices top-K nearest training rows in SHAP space
elapsed_ms per-stage timings

Fired rules are capped, not exhaustive — and the ranking lens is yours

A busy row can fire dozens of rules — many high-confidence but tiny-coverage slivers that read as noise. justify(..., top_k_rules=5) (the default) keeps only the strongest; n_fired_rules_total preserves the honest pre-cap count. Pass top_k_rules=0 to keep every fired rule.

rule_sort picks the metric the cap ranks by (default "confidence", which the platform's Score page uses):

  • "confidence" — rule precision P(class|antecedent). When the rules were measured on held-out rows it ranks by the lower bound of the Test precision's Wilson interval (precision_lower_holdout), so a rule that is 3/3 on a handful of test rows does not outrank one that is 450/500; a fired rule that covers no held-out row sorts last. Without held-out rows it ranks by the Train precision. Tie-break support.
  • "support" — evidence mass P(antecedent∩class), tie-break confidence.
  • "lift" — confidence / class base-rate ("gaining power"); the only key comparable across different consequent classes. A rule without a lift sorts last; tie-break support.
  • "coverage" — antecedent reach P(antecedent), tie-break confidence.

support, lift and coverage are Train numbers. All keys sort descending and stably; an unrecognised value falls back to "confidence".

Pipeline mode: the deployed model scores, a base tree explains

In pipeline mode the deployed model (the fit's strongest) produces prediction / decision_threshold, but shap_drivers, matched_archetype, fired_rules and nearest_indices are computed on the best introspectable base tree (the same model the profile's global artifacts were fit on). This keeps per-row SHAP cheap and consistent with the global explanation even when the deployed model is an ensemble or foundation model.

Many rows at once: justify_batch

To explain a dataset rather than a case — per-row reason codes for a scoring job — use justify_batch. It runs the stages that depend only on the row (encode, predict, label, SHAP drivers, fired rules) once over the whole frame: one pass of the feature pipeline (every domain encoder and text vectoriser), one model call, one SHAP call and one vectorised evaluation per rule. Row i of the result carries what justify(frame.iloc[[i]]) returns for the prediction, probabilities, decision threshold, drivers and fired rules:

cases = justifier.justify_batch(
    frame,               # any number of rows
    calibrated=True,
    top_k_shap=10,
    top_k_rules=5,
)
Details: what justify_batch leaves out, and how failures are handled

What stays per case:

  • Archetype, neighbours, ICE / LOCO / counterfactual are not computed — an archetype space can place a query relative to the other queries transformed with it (a UMAP space does), so only justify places a case. The fields carry the not-placed default (nearest_cluster == -1, matched_archetype is None, no neighbours).
  • SHAP from a kernel that depends on the batch is computed row by row. Most kernels explain a row the same whichever rows share the call; CatBoost's native ShapValues does not for a model trained with monotone constraints (it re-derives the trees' expected values from the rows handed in). justify_batch checks the batch's first row against that row explained alone and, when they differ, explains SHAP one row at a time — a row's drivers never depend on its batch-mates. On such a model SHAP, not the pipeline, sets the pace.

Failures stay with their rows: a SHAP or rules failure over the batch is retried row by row, so only the rows that fail again come back without drivers / rules, as from justify. A failure to encode or score the batch raises — re-explain that batch's rows one at a time to pin it on the row that causes it (the platform's reason-code export does exactly this).

Local sensitivity primitives

Three local sensitivity checks are off by default; turn them on per call. They measure how the prediction responds to perturbing this one row:

ce = result.justify(
    x_raw=row_df,
    include_ice=True,            # individual conditional expectation curves
    include_loco=True,           # leave-one-covariate-out attribution
    include_counterfactual=True, # archetype-scoped counterfactual
)
ce.ice_curves                # list[IceCurve]
ce.loco_scores               # list[LocoScore]
ce.archetype_counterfactual  # ArchetypeCounterfactual | None

These are also available as standalone functions for batch / offline use — see the local sensitivity reference:

from parleml_mli import compute_ice_for_case, compute_loco_for_case, compute_archetype_counterfactual

In the platform

The platform's per-case inference surface binds one Justifier per loaded result and calls justify per row; its batch-scoring reason-code export calls justify_batch a chunk (the job's chunk_size) at a time. Set ProfilingConfig(report=ReportConfig(output_dir="out", persist_inference_data=True)) when profiling so the train/valid/test frames are materialised next to the joblib and rows can be scored without re-running the pipeline. The frames are written under out/model_training_data/; setting the persistence flag without output_dir does not write them.

Runnable demo

examples/demos/demo_decision_support.py is the end-to-end script: it profiles a model with the typed ProfilingConfig surface, saves + reloads the result, then binds a Justifier and justifies held-out rows — all with the public API and no backend. Run it with python demo_decision_support.py --fast.

Next