ProfilingResult¶
The central hub returned by fit_profiler. Holds the
archetype artifacts, exposes the per-case justifier / justify, plot
helpers, and save/load. save stamps the joblib with
parleml_mli.result.PROFILING_RESULT_SCHEMA_VERSION; load refuses any other stamp
(or an unstamped pickle) with a parleml_mli.StaleArtifactError (a
ValueError) that says to refit the profile with the installed mli. Configure optional stages eagerly via
ProfilingConfig sections, or populate them lazily
afterwards with the build_*_result functions documented below
(build_rules_result(result, ...) and friends — these are module-level
functions that take the result as their first argument, not methods on
ProfilingResult). See the Profiling guide.
In short
ProfilingResultholds the archetypes and every optional artifact, the plot helpers and the per-casejustifier/justify.result.capabilitiessays what the backend could produce;result.resolve_capabilities()gives a uniform view.- The
build_*_resultfunctions fill in an optional stage later. They are module-level functions, not methods. save/loaduse one joblib with a schema stamp; another stamp raisesStaleArtifactError.
ProfilingResult
dataclass
¶
ProfilingResult(
labels_: ndarray,
summary_df: DataFrame,
medoids_df: DataFrame,
characterization_: dict[int, DataFrame],
clustering_result: ClusteringResult,
space_type: SpaceType,
task_type: TaskType,
contamination_df: DataFrame | None = None,
umap_embedding_: ndarray | None = None,
cluster_thresholds_: dict[int, float] = dict(),
base_target_rate_: float = 0.0,
min_odds_ratio_: float = 3.0,
min_stability_: float = DEFAULT_MIN_STABILITY,
stability_: StabilityResult | None = None,
llm_analysis_: dict[str, object] | None = None,
rules_: (
RulesResult | MulticlassRulesResult | None
) = None,
error_analysis_: ErrorAnalysisResult | None = None,
model_evaluation_: "ModelEvaluationResult | None" = None,
independent_rules_: "IndependentRuleModelResult | None" = None,
tree_diagrams_: list[str] | None = None,
explanations_: ExplanationsResult | None = None,
eda_: "EDAResult | None" = None,
target_: ResolvedTarget | None = None,
threshold_: ResolvedThreshold | None = None,
_space: DistanceSpace | None = None,
_target_data_t: ndarray | None = None,
_medoid_data_t: ndarray | None = None,
_target_shap_values: ndarray | None = None,
_shap_class_index: int | None = None,
_predictions_target: ndarray | None = None,
_feature_names: list[str] = list(),
_model: object | None = None,
_model_type: ModelType | None = None,
_umap_sample_idx: ndarray | None = None,
_nn_index: NeighborIndex | None = None,
_nn_lock: Lock = Lock(),
_nn_index_original: "OriginalFeatureIndex | None" = None,
_nn_lock_original: Lock = Lock(),
_X_target: DataFrame | None = None,
_X_target_orig: DataFrame | None = None,
_target_mask: ndarray | None = None,
_y_full: ndarray | None = None,
_id_columns: list[str] | None = None,
_encoding_registry: EncodingRegistry = (
lambda: IdentityRegistry()
)(),
_n_jobs: int = -1,
meta: dict[str, Any] = dict(),
config: "RunSettings | None" = None,
narration_: "NarrationBundle | None" = None,
_classes: "dict[str, ClassProfile] | None" = None,
_submodel_results: "dict[str, ProfilingResult] | None" = None,
capabilities: "Capabilities | None" = None,
)
Bases: _ProfilingResultPlotsMixin, _RetrievalMixin, _InferenceMixin, _MulticlassMixin, _ReportSerializationMixin
Container for all profiling outputs with retrieval and plotting methods.
decision_threshold
property
¶
The resolved binary cut — :attr:threshold_'s value.
has_predictions
property
¶
True when model scores are available in _predictions_target (enables plot_scores).
evaluation_
property
¶
Per-split holdout evaluations, keyed by split name.
Returns {split_name: BinaryEvaluation | MulticlassEvaluation |
RegressionSplitEvaluation}. Convenience alias for
self.model_evaluation_.splits (with empty dict fallback).
model_
property
¶
The fitted model used for profiling (None if not stored).
Stored when fit_profiler is called with include_model=True
(the default for the native path). Needed by SHAP / surrogate /
permutation-importance recomputation and by the justification path.
model_type_
property
¶
Detected :class:~parleml_mli.core.types.ModelType of :attr:model_.
feature_names_
property
¶
Encoded feature names aligned with :attr:target_features_ columns.
target_predictions_
property
¶
Model scores for the target cohort (None when unavailable).
Probabilities for the positive class on classifiers, raw model
output on regressors — aligned row-for-row with :attr:labels_.
target_shap_values_
property
¶
(n_target_rows, n_features) SHAP matrix for the target cohort.
None when the profile was not fit with a SHAP-aware space.
Aligned row-for-row with :attr:labels_ and column-for-column
with :attr:feature_names_.
target_features_
property
¶
Encoded target-cohort feature matrix (None if not stored).
The model-space frame the profile was built from. For the
human-readable original values see :attr:target_features_original_.
target_features_original_
property
¶
Original-space (human-readable) target-cohort features.
Populated only when X_target_orig= was passed to
fit_profiler; None otherwise.
target_mask_
property
¶
Boolean mask selecting target rows from the full fit-time X.
None when the profile was fit directly on a target-only matrix.
id_columns_
property
¶
Original columns pinned into representative examples (e.g. record IDs).
umap_sample_index_
property
¶
Row index of the UMAP scatter sample, or None when not downsampled.
When the run downsampled the 2-D scatter, :attr:umap_embedding_ covers
only these rows; use this index to align :attr:labels_ /
:attr:target_predictions_ to the embedding. None means the
embedding is row-aligned with the full target cohort.
encoding_registry_
property
¶
encoding_registry_: EncodingRegistry
Categorical-encoding registry used to decode engineered features.
Defaults to :class:~parleml_mli.core.encoding.IdentityRegistry (no decoding)
when the profile was fit without categorical encoders.
resolve_capabilities ¶
resolve_capabilities() -> Capabilities
Return self.capabilities when stamped, else derive from
field presence.
The derivation also seeds :func:parleml_mli.fit_profiler's own stamp, so
consumers get a uniform Capabilities view across native and lite
artifacts and the UI / AutoDoc need no second code path.
get_cluster_features ¶
Return the SHAP / z-score characterization table for one cluster.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cluster_id
|
int
|
Cluster label. Use |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Columns depend on whether SHAP was available at fit time:
Rows are sorted by importance (descending). |
Raises:
| Type | Description |
|---|---|
KeyError
|
If cluster_id does not exist in the characterization. |
save ¶
Persist this result to disk via :func:joblib.dump.
Persists alongside the pickled result a small metadata header
recording :data:PROFILING_RESULT_SCHEMA_VERSION, the mli
version, the RUN_SETTINGS_SCHEMA_VERSION, and the installed
versions of the stochastic libraries the artifact depends on
(scikit-learn, umap-learn, hdbscan, shap).
:meth:load refuses any other schema stamp and emits a
:class:UserWarning when the mli or library versions differ.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
File path (recommended extension: |
required |
include_model
|
bool
|
When |
True
|
load
classmethod
¶
load(path: str) -> ProfilingResult
Load a result written by :meth:save.
Returns:
| Type | Description |
|---|---|
ProfilingResult
|
Fully restored result object with all methods available.
If the result was saved with |
Raises:
| Type | Description |
|---|---|
StaleArtifactError
|
The joblib carries no schema stamp or one other than
:data: |
TypeError
|
The envelope holds something other than a |
Notes
The active mli version and the installed versions of the
stochastic libraries (scikit-learn, umap-learn,
hdbscan, shap) are checked against the recorded header; a
:class:UserWarning is emitted per mismatch.
Capabilities¶
The flag matrix stamped on every result (result.capabilities) by
fit_profiler (native) and monitor_predictions (lite) so consumers can
grey out surfaces the underlying model can't power. Use
ProfilingResult.resolve_capabilities for a
uniform view across native and lite artifacts (an unstamped SHAP-only run
derives its flags from field presence).
Capabilities
dataclass
¶
Capabilities(
model_evaluation: bool = False,
archetypes: bool = False,
rules: bool = False,
surrogate: bool = False,
error_tree: bool = False,
drift: bool = False,
fairness: bool = False,
shap: bool = False,
pdp_ice: bool = False,
counterfactual: bool = False,
eda: bool = False,
leakage: bool = False,
backend: Literal[
"native", "lite_model", "lite_predictions"
] = "native",
notes: str = "",
)
Flag matrix indicating which downstream surfaces are populated.
Stamped by :func:parleml_mli.fit_profiler (backend="native", when a
recognised model was supplied) and :func:parleml_mli.monitor_predictions
(backend="lite_model" / "lite_predictions") so consumers (UI,
AutoDoc, backend wrappers) can grey out tabs the underlying model can't
power. An unstamped result (a SHAP-only native run) derives its flags
from field presence — see :meth:ProfilingResult.resolve_capabilities.
Compute-on-demand builders¶
Populate an optional stage on an already-fitted result, e.g.
result.rules_ = build_rules_result(result, X=X_train). Each is the lazy
peer of the eager ProfilingConfig section with the same name; all are
re-exported from the top level (from parleml_mli import build_rules_result).
build_rules_result ¶
build_rules_result(
result: ProfilingResult,
config: RulesConfig | None = None,
*,
X: DataFrame | None = None,
y: ndarray | Series | None = None,
X_holdout: DataFrame | None = None,
y_holdout: ndarray | Series | None = None,
holdout_basis: HoldoutBasis | None = None,
random_state: int = 42,
verbose: bool = False,
allow_target_mask_as_y: bool = False
) -> RulesResult | MulticlassRulesResult
Extract one global ruleset from the stored model via rule extraction.
Runs a single extraction pass over the full (X, y) matrix and,
when cluster labels are available, derives per-archetype rule-
firing fingerprints from that ruleset (see
:func:parleml_mli.archetypes.fingerprints.compute_archetype_fingerprints). Each
archetype is then characterised by which of the global rules
fire on its members — a single source of truth for rule logic,
no duplicate LLM translation work, and a natural cross-archetype
comparison surface.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
RulesConfig | None
|
Typed :class: :attr:
A fan-out ( See :func: |
None
|
X
|
DataFrame | None
|
Full feature matrix (target + reference rows). Required — re-load the source dataset and pass it in; it is never stored on the result. |
None
|
y
|
ndarray | Series | None
|
Optional binary label vector aligned with |
None
|
X_holdout
|
DataFrame | None
|
Held-out rows — rows the rule search never saw, and ideally the model
neither ( |
None
|
y_holdout
|
DataFrame | None
|
Held-out rows — rows the rule search never saw, and ideally the model
neither ( |
None
|
holdout_basis
|
HoldoutBasis | None
|
Where the held-out rows come from, stamped on every rule set. |
None
|
allow_target_mask_as_y
|
bool
|
Opt-in to using the stored |
False
|
random_state
|
int
|
Runtime knobs forwarded to :func: |
42
|
verbose
|
int
|
Runtime knobs forwarded to :func: |
42
|
Returns:
| Type | Description |
|---|---|
RulesResult | MulticlassRulesResult
|
A binary/regression run returns a :class: |
Raises:
| Type | Description |
|---|---|
ValueError
|
|
TypeError
|
Model is not all-class-engine compatible. |
ImportError
|
|
build_explanations_result ¶
build_explanations_result(
result: ProfilingResult,
config: ExplanationsConfig | None = None,
*,
X: DataFrame | None = None,
y: ndarray | Series | None = None,
ovr_classes: Any | None = None,
decision_threshold: (
ResolvedThreshold | float | None
) = None
) -> ExplanationsResult
Compute every requested global-model interpretability artifact.
Wraps :func:parleml_mli.explanations.explain with result's stored model
and full-population matrices, attaching per-archetype overlays when
result.labels_ is available. Returns the global
:class:ExplanationsResult; the caller assigns it to
result.explanations_ (fit_profiler does this during the
one-pass build). When ovr_classes is set, per-class slots on
result.classes are additionally populated in place.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
ExplanationsConfig | None
|
Typed :class: |
None
|
X
|
DataFrame | None
|
Full evaluation matrix (never stored on the result — re-load the source dataset and pass it in). Required. |
None
|
y
|
ndarray | Series | None
|
Optional label vector; defaults to |
None
|
ovr_classes
|
Any | None
|
When set on a multiclass classifier
(
Compacity is also fanned out per-class when requested (it reuses
each :class: |
None
|
decision_threshold
|
ResolvedThreshold | float | None
|
Overrides the binary cut-off forwarded to the threshold-aware
artifacts. |
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
|
Returns:
| Type | Description |
|---|---|
ExplanationsResult
|
The global artifacts; assign to |
build_eda_result ¶
build_eda_result(
result: ProfilingResult, **kw: Any
) -> "EDAResult"
Compute every requested EDA artifact for result.
Wraps :func:parleml_mli.eda.compute_eda with the supplied full feature
matrix and optional ground truth. Returns the :class:EDAResult;
the caller assigns it to result.eda_ (fit_profiler does this
during the one-pass build).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
result
|
ProfilingResult
|
The profiling result; only |
required |
**kw
|
Any
|
Forwarded verbatim to :func: |
{}
|
Raises:
| Type | Description |
|---|---|
ValueError
|
The caller did not pass |
Returns:
| Type | Description |
|---|---|
EDAResult
|
The EDA artifacts; assign to |
build_model_evaluation_result ¶
build_model_evaluation_result(
result: ProfilingResult,
*,
splits: (
Mapping[str, tuple[DataFrame, ndarray | Series]]
| None
) = None,
X_train: DataFrame | None = None,
y_train: ndarray | Series | None = None,
decision_threshold: (
ResolvedThreshold | float | None
) = None,
n_bootstrap: int = 0,
ci_alpha: float = 0.05,
multiclass_threshold_metric: str = "f1"
) -> "ModelEvaluationResult"
Score holdout splits and build the evaluation artifact for result.
Returns the :class:ModelEvaluationResult; the caller assigns it to
result.model_evaluation_ (fit_profiler does this during the
one-pass build).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
result
|
ProfilingResult
|
The profiling result carrying the fitted model. |
required |
splits
|
Mapping[str, tuple[DataFrame, ndarray | Series]] | None
|
Optional |
None
|
X_train
|
DataFrame | None
|
Training matrix used for profiling. Included in the evaluation bundle when holdout splits are provided. |
None
|
y_train
|
DataFrame | None
|
Training matrix used for profiling. Included in the evaluation bundle when holdout splits are provided. |
None
|
decision_threshold
|
ResolvedThreshold | float | None
|
Classification cut-off for fixed-threshold metrics. Defaults to the
run's |
None
|
n_bootstrap
|
int
|
Percentile-bootstrap confidence intervals for the headline metrics,
populating each split's |
0
|
ci_alpha
|
int
|
Percentile-bootstrap confidence intervals for the headline metrics,
populating each split's |
0
|
multiclass_threshold_metric
|
str
|
F-beta spec for the per-class OvR threshold tuning that happens on the
first split ( |
'f1'
|
build_stability_result ¶
build_stability_result(
result: ProfilingResult,
*,
n_bootstrap: int = DEFAULT_N_BOOTSTRAP,
min_cluster_size: int | None = None,
min_samples: int | None = None,
cluster_selection_method: (
ClusterSelectionMethod | None
) = None,
n_jobs: int = -1,
random_state: int = 42
) -> StabilityResult
Assess cluster stability via bootstrap resampling.
Resamples the target data with replacement n_bootstrap times,
re-runs HDBSCAN with the same parameters, and measures the
best-match Jaccard similarity between the original clusters and
each bootstrap's clusters. Returns the :class:StabilityResult;
the caller assigns it to result.stability_.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
result
|
ProfilingResult
|
The profiling result whose clustering is re-sampled. |
required |
n_bootstrap
|
int
|
Number of bootstrap iterations (default 20). |
DEFAULT_N_BOOTSTRAP
|
min_cluster_size
|
int | None
|
HDBSCAN |
None
|
min_samples
|
int | None
|
HDBSCAN |
None
|
cluster_selection_method
|
ClusterSelectionMethod | None
|
Defaults to the original clustering method. |
None
|
n_jobs
|
int
|
Parallel workers for HDBSCAN. |
-1
|
random_state
|
int
|
Seed for bootstrap sampling. |
42
|
Returns:
| Type | Description |
|---|---|
StabilityResult
|
Per-cluster Jaccard scores and an overall weighted mean. |
build_error_analysis_result ¶
build_error_analysis_result(
result: ProfilingResult,
*,
X: DataFrame | None = None,
target_mask: ndarray | Series | None = None,
shap_full: ndarray | None = None,
shap_cache_path: str | None = None,
recompute_shap_on_cache_miss: bool = False,
y_true: ndarray | Series | None = None,
scores: ndarray | Series | None = None,
archetype_labels: ndarray | None = None,
threshold: (
ResolvedThreshold | float | str | None
) = None,
assignment: str = "territory"
) -> ErrorAnalysisResult
Compute per-archetype classification error metrics.
Three invocation modes are supported, listed from highest- to lowest-level:
Auto full-population mode (recommended)
Pass X (and optionally target_mask) — same arrays you
gave to :func:fit_profiler. The library handles everything:
it loads SHAP from shap_cache_path (or auto-computes via the
stored model), keeps each target row in its archetype
(labels_), places every reference row on the archetypes by
the assignment rule, uses the model to score reference rows,
and combines with the stored target predictions. Produces a
proper confusion matrix (TP/FP/TN/FN) per archetype at the run's
resolved cut (result.threshold_.cut).
Manual full-population mode Pre-compute everything yourself and pass y_true, scores, and archetype_labels aligned to the complete dataset. Useful when scores or archetype labels come from a custom pipeline.
Detection-rate mode (zero-config fallback) Don't pass any extra data. Uses only the target-class cases stored on the result, at the same cut. Since FP = 0 by construction, the meaningful metric is recall (= "what fraction of this archetype does the model detect?"). Charts are labelled "Model Detection Rate" to make this explicit.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
result
|
ProfilingResult
|
The profiling result carrying the fitted model, stored target predictions/labels, archetype labels, and space. |
required |
X
|
DataFrame | None
|
Full feature matrix (target + reference rows) — same one passed
to :func: |
None
|
target_mask
|
ndarray | Series | None
|
Boolean mask aligning X with target rows. Defaults to the
stored |
None
|
shap_full
|
ndarray | None
|
Pre-computed SHAP values for X (full population). Optional —
when |
None
|
shap_cache_path
|
str | None
|
Path to a |
None
|
recompute_shap_on_cache_miss
|
bool
|
When |
False
|
y_true
|
ndarray | Series | None
|
Ground-truth labels (manual full-population mode), binarised
against the run's positive class (
|
None
|
scores
|
ndarray | Series | None
|
Continuous model probability scores (manual full-population
mode). Provide aligned with y_true and archetype_labels.
When |
None
|
archetype_labels
|
ndarray | None
|
Archetype assignment for each case in the full population
(manual full-population mode). Obtain with:: |
None
|
threshold
|
ResolvedThreshold | float | str | None
|
The cut that binarises scores into predictions. |
None
|
assignment
|
str
|
How reference cases are placed on archetypes in auto
full-population mode (see
:mod:
|
'territory'
|
Returns:
| Type | Description |
|---|---|
ErrorAnalysisResult
|
Assign to |
Raises:
| Type | Description |
|---|---|
ValueError
|
When the requested mode cannot be satisfied (missing model, inconsistent array lengths, missing target_mask, etc.). |
build_compacity_result ¶
build_compacity_result(
result: ProfilingResult,
*,
approx: float = 0.9,
fixed_k: int = 5
) -> CompacityResult
Compute per-archetype SHAP compacity from result's target SHAP.
"How few features does the model need to explain a typical row in
each archetype?". Aggregates
:func:parleml_mli.explanations.compacity.compute_compacity over the rows in
result and stores the output on result.explanations_.compacity
(creating result.explanations_ when absent), returning it too.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
result
|
ProfilingResult
|
The profiling result carrying target SHAP values and labels. |
required |
approx
|
float
|
Cumulative-fraction threshold in |
0.9
|
fixed_k
|
int
|
Top-k headline used for the right-bar in
:meth: |
5
|
Returns:
| Type | Description |
|---|---|
CompacityResult
|
Also stored on |
Raises:
| Type | Description |
|---|---|
ValueError
|
When SHAP values are not available on this result. |
Archetype output types¶
The typed artifacts hanging off a profiled result — the per-cluster
signatures, bootstrap stability, error analysis, similar-case retrieval, and
per-archetype rule-firing fingerprints. Error analysis cuts at the run's
result.threshold_ (its threshold_source is that threshold's source) and
counts a target row in the archetype its clustering label names, as
summary_df and contamination_df do; only reference rows are placed by the
nearest-medoid rule.
ClusterSignature
dataclass
¶
ClusterSignature(
cluster_id: int,
n_members: int,
shap_profile: DataFrame,
numeric_stats: DataFrame,
categorical_stats: dict[str, DataFrame] = dict(),
top_features: list[str] = list(),
cluster_meta: dict[str, Any] = dict(),
missing_in_target: list[str] = list(),
)
Archetype signature combining SHAP importance with actual feature distributions.
Attributes:
| Name | Type | Description |
|---|---|---|
cluster_id |
int
|
Cluster label (-1 = noise). |
n_members |
int
|
Number of target cases in this cluster. |
shap_profile |
DataFrame
|
Top features ranked by |
numeric_stats |
DataFrame
|
Cluster vs all-target statistics for numeric top features.
Columns: |
categorical_stats |
dict[str, DataFrame]
|
For each categorical top feature: DataFrame with columns
|
top_features |
list[str]
|
Ordered list of top feature names (same order as |
cluster_meta |
dict[str, Any]
|
Optional cluster-level metadata dict populated by
|
missing_in_target |
list[str]
|
Top features whose target-population column is entirely NaN, so no
distribution exists to describe. They still rank by importance —
on a native-NaN model their missingness is what the model split
on — and are rendered |
CombinedClusterSignature
dataclass
¶
CombinedClusterSignature(
cluster_id: int,
n_members: int,
original_signature: ClusterSignature,
engineered_signature: ClusterSignature,
combined_top_features: list[str] = list(),
feature_source_map: dict[str, str] = dict(),
derivative_map: dict[str, list[str]] = dict(),
cluster_meta: dict[str, Any] = dict(),
representative_examples: DataFrame | None = None,
shared_original_extras: list[str] = list(),
)
Archetype signature combining original and engineered feature views.
Attributes:
| Name | Type | Description |
|---|---|---|
cluster_id |
int
|
Cluster label. |
n_members |
int
|
Number of target cases in this cluster. |
original_signature |
ClusterSignature
|
Signature built from the original (clean, business-readable) dataset. |
engineered_signature |
ClusterSignature
|
Signature built from the feature-engineered dataset. |
combined_top_features |
list[str]
|
Ordered feature list for the combined plot: original features first (by aggregated SHAP), then engineered-only features. Shared features that are identical in both datasets appear once. |
feature_source_map |
dict[str, str]
|
Maps each feature name in combined_top_features to |
derivative_map |
dict[str, list[str]]
|
Maps each original feature name to the list of engineered features
derived from it (via the encoding registry's |
cluster_meta |
dict[str, Any]
|
Cluster-level metadata (prevalence, purity, coverage, odds ratio).
Populated by |
representative_examples |
DataFrame | None
|
Small DataFrame of sample cluster members' original feature values.
|
shared_original_extras |
list[str]
|
Features present identically in both frames that rank into the
engineered top-N but not the original top-N. They are displayed
once, tagged |
StabilityResult
dataclass
¶
StabilityResult(
per_cluster_jaccard: dict[int, float],
overall_jaccard: float,
n_bootstrap: int,
cluster_sizes: dict[int, int] = dict(),
n_failed: int = 0,
n_success: int = 0,
)
Per-cluster stability scores from bootstrap resampling.
Attributes:
| Name | Type | Description |
|---|---|---|
per_cluster_jaccard |
dict[int, float]
|
Mean Jaccard similarity per original cluster (higher = more stable). |
overall_jaccard |
float
|
Weighted mean Jaccard across all clusters. |
n_bootstrap |
int
|
Number of bootstrap iterations requested. |
cluster_sizes |
dict[int, int]
|
Original cluster sizes used for weighting. |
n_failed |
int
|
Number of bootstrap iterations whose re-clustering raised and were excluded from the means. |
n_success |
int
|
Number of bootstrap iterations that produced a clustering and
contributed to |
ErrorAnalysisResult
dataclass
¶
ErrorAnalysisResult(
by_archetype: DataFrame,
n_evaluated: int,
threshold: float,
positive_label: Any = None,
mode: ErrorAnalysisMode = "detection_rate",
threshold_source: ThresholdSource = "explicit",
overall_f1_at_threshold: float | None = None,
scores_full: ndarray | None = None,
archetype_labels_full: ndarray | None = None,
schema_version: int = ERROR_ANALYSIS_SCHEMA_VERSION,
)
Per-archetype classification error metrics.
Attributes:
| Name | Type | Description |
|---|---|---|
by_archetype |
DataFrame
|
DataFrame with one row per
|
n_evaluated |
int
|
Total number of cases used in the computation. |
threshold |
float
|
The cut on the positive-class score that derived |
positive_label |
Any
|
Display label of the positive class (the run's
|
mode |
ErrorAnalysisMode
|
|
threshold_source |
ThresholdSource
|
Where threshold came from: the
:data: |
overall_f1_at_threshold |
float | None
|
Binary F1 of the positive class achieved on the full evaluated
population at threshold. |
scores_full |
ndarray | None
|
|
archetype_labels_full |
ndarray | None
|
|
schema_version |
int
|
:data: |
RetrievalResult
dataclass
¶
RetrievalResult(
nearest_cluster: int | None,
is_novel: bool,
nearest_indices: ndarray,
nearest_distances: ndarray,
distance_to_nearest_medoid: float,
cluster_threshold: float,
explanation: DataFrame | None = None,
nearest_original_features: DataFrame | None = None,
)
Similar historical cases for one query, plus its archetype assignment.
nearest_cluster is the nearest archetype (None only when the
profile has none); is_novel means the query lies outside that
archetype's radius, or the archetype has no radius; cluster_threshold
is that radius (NaN when none is stored). The assignment follows
:mod:parleml_mli.archetypes.profiling.assignment's ball rule.
retrieve_new_cases_original_space computes no assignment and leaves
these fields at None / False / NaN.
FingerprintResult
dataclass
¶
FingerprintResult(
rule_ids: list[int] = list(),
cluster_ids: list[int] = list(),
fire_rate: ndarray = (
lambda: zeros((0, 0), dtype=float)
)(),
specificity: ndarray = (
lambda: zeros((0, 0), dtype=float)
)(),
rule_total_matches: ndarray = (
lambda: zeros((0,), dtype=int)
)(),
cluster_sizes: ndarray = (
lambda: zeros((0,), dtype=int)
)(),
n_total: int = 0,
)
Rule-firing fingerprints across archetypes.
Attributes:
| Name | Type | Description |
|---|---|---|
rule_ids |
list[int]
|
|
cluster_ids |
list[int]
|
Sorted cluster ids included in the fingerprint (drops |
fire_rate |
ndarray
|
|
specificity |
ndarray
|
|
rule_total_matches |
ndarray
|
|
cluster_sizes |
ndarray
|
|
n_total |
int
|
Total number of rows in |
for_cluster ¶
for_cluster(
cluster_id: int,
*,
min_fire_rate: float = _DEFAULT_FIRE_RATE_THRESHOLD,
min_specificity: float = 0.0,
sort_by: str = "fire_rate"
) -> DataFrame
Return a per-rule DataFrame characterising one archetype.
Columns: rule_id, fire_rate, specificity,
n_matches_in_cluster, rule_total_matches. Sorted by
sort_by ("fire_rate" default; "specificity" to
surface rules unique to this archetype).
for_rule ¶
for_rule(
rule_id: int,
*,
min_fire_rate: float = 0.0,
min_specificity: float = _DEFAULT_SPECIFICITY_THRESHOLD,
sort_by: str = "specificity"
) -> DataFrame
Return a per-cluster DataFrame for one rule.
Inverse of :meth:for_cluster — surfaces which archetypes a
given rule characterises, sorted by specificity by default.
Result payload types¶
Returned by the retrieval / reporting surfaces — predict_archetype
returns an ArchetypePrediction, generate_report returns a ReportResult.
ArchetypePrediction
dataclass
¶
ArchetypePrediction(
labels: ndarray,
nearest_cluster: ndarray,
distance_ratio: ndarray,
distances: ndarray,
)
Rich archetype assignment with soft confidence.
Attributes:
| Name | Type | Description |
|---|---|---|
labels |
ndarray
|
Assigned archetype id per case. |
nearest_cluster |
ndarray
|
Nearest archetype per case regardless of radius; |
distance_ratio |
ndarray
|
|
distances |
ndarray
|
Raw distance to the nearest medoid per case ( |
ReportResult
dataclass
¶
ReportResult(
output_dir: str,
report_path: str,
llm_report_path: str | None,
report: dict[str, object],
figure_paths: dict[str, str],
)
Return type for :meth:ProfilingResult.generate_report.
Per-class profile¶
The lightweight per-class slice exposed by result.classes[class_key] /
result.profile_for(...) — binary runs carry one entry, multiclass runs one
per class (the one-vs-rest fan-out).
ClassProfile
dataclass
¶
ClassProfile(
class_label: Any,
class_key: str,
target_mask: ndarray,
base_target_rate: float,
synthetic: bool = False,
labels_: ndarray | None = None,
clustering_result: "ClusteringResult | None" = None,
summary_df: DataFrame | None = None,
contamination_df: DataFrame | None = None,
medoids_df: DataFrame | None = None,
characterization_: dict[int, DataFrame] = dict(),
umap_embedding_: ndarray | None = None,
cluster_thresholds_: dict[int, float] = dict(),
stability_: "StabilityResult | None" = None,
min_odds_ratio_: float = 3.0,
min_stability_: float = DEFAULT_MIN_STABILITY,
error_analysis_: "ErrorAnalysisResult | None" = None,
rules: dict["RuleEngine", "RuleSetSummary"] = dict(),
rule_fingerprints: dict["RuleEngine", Any] = dict(),
fairness: dict[str, "FairnessResult"] = dict(),
pdp: dict[str, "PDPResult"] = dict(),
ale: dict[str, "ALEResult"] = dict(),
permutation_importance: "PermutationImportanceResult | None" = None,
error_tree: "ErrorTreeResult | None" = None,
psi: "PSIResult | None" = None,
compacity: "CompacityResult | None" = None,
_space: "DistanceSpace | None" = None,
_target_data_t: ndarray | None = None,
_medoid_data_t: ndarray | None = None,
_nn_index: Any | None = None,
_nn_index_original: Any | None = None,
_X_target: DataFrame | None = None,
_X_target_orig: DataFrame | None = None,
_predictions_target: ndarray | None = None,
_target_shap_values: ndarray | None = None,
)
Per-case justification¶
Bind a model with ProfilingResult.justifier and
call justify per row, or justify_batch to explain a whole frame at the cost of
one batch — see the decision-support guide.
Justifier ¶
Model-bound justification engine — see :meth:ProfilingResult.justifier.
Obtain one via result.justifier(pipeline) rather than constructing it
directly. The adapter and the active per-class view are resolved once at
construction; :meth:justify is then cheap to call per row, and
:meth:justify_batch explains many rows for the fixed cost of one.
justify ¶
justify(
x_raw: DataFrame,
*,
calibrated: bool = True,
active_class: Any = None,
top_k_shap: int = 10,
top_k_neighbors: int = 5,
top_k_rules: int = 5,
rule_sort: str = "confidence",
include_shap: bool = True,
include_rules: bool = True,
include_neighbors: bool = True,
include_ice: bool = False,
include_loco: bool = False,
include_counterfactual: bool = False,
top_k_ice: int = 3,
top_k_loco: int = 10,
counterfactual_target_archetype: int | None = None
) -> "CaseExplanation"
Justify a single prediction → a :class:parleml_mli._dsd_types.CaseExplanation.
Composes the predict / SHAP / archetype / rules / nearest-neighbour primitives into a structured decision-support payload — the cold-path object an analyst-facing DB / viewer consumes after the production hot path has already returned a fast score.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x_raw
|
DataFrame
|
Single-row raw input frame (1 × p). For sklearn-mode this must already be in engineered-feature shape; for pipeline-mode it can be the raw frame the bound pipeline transforms internally. |
required |
calibrated
|
bool
|
When True, request the calibrated probability flavour from the
pipeline. Ignored for sklearn-mode (no calibrator) and for
regression. The returned |
True
|
active_class
|
Any
|
Class label whose probability column drives prediction labelling
and SHAP slicing — as published on |
None
|
top_k_shap
|
int
|
See :class: |
10
|
top_k_neighbors
|
int
|
See :class: |
10
|
top_k_rules
|
int
|
Max fired rules to keep after ranking by |
5
|
rule_sort
|
str
|
Which metric orders fired rules before the |
'confidence'
|
include_shap
|
bool
|
Gating flags. The hot path normally calls this with everything on;
a worker that has already paid for SHAP elsewhere can pass
|
True
|
include_rules
|
bool
|
Gating flags. The hot path normally calls this with everything on;
a worker that has already paid for SHAP elsewhere can pass
|
True
|
include_neighbors
|
bool
|
Gating flags. The hot path normally calls this with everything on;
a worker that has already paid for SHAP elsewhere can pass
|
True
|
include_ice
|
bool
|
Stage-2 sensitivity flags (see
|
False
|
include_loco
|
bool
|
Stage-2 sensitivity flags (see
|
False
|
include_counterfactual
|
bool
|
Stage-2 sensitivity flags (see
|
False
|
top_k_ice
|
int
|
Trim the corresponding outputs. |
3
|
top_k_loco
|
int
|
Trim the corresponding outputs. |
3
|
counterfactual_target_archetype
|
int | None
|
Explicit override for the safer-archetype picker. |
None
|
justify_batch ¶
justify_batch(
x_raw: DataFrame,
*,
calibrated: bool = True,
active_class: Any = None,
top_k_shap: int = 10,
top_k_rules: int = 5,
rule_sort: str = "confidence",
include_shap: bool = True,
include_rules: bool = True
) -> list["CaseExplanation"]
Justify every row of x_raw at once → one :class:parleml_mli._dsd_types.CaseExplanation per row.
Row i carries what justify(x_raw.iloc[[i]]) returns with the same
arguments for the prediction, probabilities, decision threshold, SHAP drivers
and fired rules. The difference is cost: the feature pipeline, the model and
SHAP each run once over the frame and every rule is evaluated once over it, so
the fixed cost of each stage (encoders, vectorisers, the SHAP kernel's setup)
is paid once per batch rather than once per row. This is the path for
explaining a dataset — e.g. per-row reason codes for a batch-scoring job.
One exception keeps the row equality honest: a SHAP kernel that explains a row
differently depending on the rows passed with it (CatBoost's, for a model
trained with monotone constraints) is detected on the batch's first row and
then run row by row, so a row's drivers never depend on its batch-mates.
The stages that place one case stay on :meth:justify: archetype assignment
(an archetype space can place a query relative to the other queries it is
transformed with), nearest neighbours and the Stage-2 sensitivity primitives
(ICE / LOCO / counterfactual). Their fields carry the not-placed default
(nearest_cluster=-1, matched_archetype=None, no neighbours), and each
stage time in elapsed_ms is the batch's, amortised per row.
A SHAP or rules failure over the batch is retried row by row, so it only costs
the rows that cause it (they come back without drivers / without rules, as a
single-row justify returns them). A failure to encode or score the batch
raises — the caller decides how to isolate the offending row (e.g. by
justifying that batch's rows one at a time).
Parameters are those of :meth:justify; x_raw may hold any number of rows
(none returns []).
lookalikes ¶
Raw-feature nearest neighbours for x_raw (model-free).
A thin passthrough to
:meth:ProfilingResult.retrieve_new_cases_original_space — the
"lookalike cases by raw feature values" lens. Needs no model call, so
it works even for a model-less profile that was saved with the original
feature matrix.
Per-case payload types¶
Returned by Justifier.justify — see
the decision-support guide.
CaseExplanation
dataclass
¶
CaseExplanation(
x_raw: DataFrame,
x_encoded: DataFrame,
proba_calibrated: ndarray | None,
proba_raw: ndarray | None,
prediction: Any,
decision_threshold: float | None,
shap_drivers: list[ShapDriver],
matched_archetype: int | None,
nearest_cluster: int,
distance_to_medoid: float,
distance_ratio: float,
is_novel: bool,
archetype_name: str | None,
archetype_severity: str | None,
fired_rules: list[FiredRule],
nearest_indices: ndarray,
nearest_distances: ndarray,
nearest_original_features: DataFrame | None,
calibration_active: bool,
inference_mode: Literal["sklearn", "pipeline"],
active_class: Any,
elapsed_ms: dict[str, float | str] = dict(),
ice_curves: list["IceCurve"] = list(),
loco_scores: list["LocoScore"] = list(),
archetype_counterfactual: "ArchetypeCounterfactual | None" = None,
n_fired_rules_total: int = 0,
)
Everything justify produces for a single row.
Construction is driven by :meth:parleml_mli._justifier.Justifier.justify
and :mod:parleml_mli._case_explanation; consumers should treat this object as
read-only.
See mli/docs/inference-and-dsd-plan.md §5.1 for the field
contract and §5.2 for the body sketch that derives each field.
ShapDriver
dataclass
¶
ShapDriver(
feature: str,
origin_feature: str,
kind: str,
value: float | str | None,
shap: float,
abs_shap_value: float,
)
One row of the top-K SHAP driver table.
Field names match the inference-and-dsd-plan §5.1 wire contract
(origin_feature + shap); kind and abs_shap_value are
additive viewer hints not required by the contract.
Attributes:
| Name | Type | Description |
|---|---|---|
feature |
str
|
Engineered feature name as the algorithm sees it. |
origin_feature |
str
|
Business-readable origin column from the encoding registry; falls
back to |
kind |
str
|
Encoding kind ( |
value |
float | str | None
|
The encoded value used at scoring time. |
shap |
float
|
Signed SHAP attribution. |
abs_shap_value |
float
|
|
FiredRule
dataclass
¶
FiredRule(
rule_id: str,
engine: str,
antecedent: str,
antecedent_components: list[dict[str, Any]],
predicted_class: int,
confidence: float,
support: float,
coverage: float,
lift: float | None = None,
precision_holdout: float | None = None,
precision_lower_holdout: float | None = None,
holds_up: HoldsUp | None = None,
)
A single rule that fired on the case.
Carries the rule-chip shape the backend projects to the UI so the
Inference page renders the rule with the same chip widgets used
everywhere else (consumers read antecedent_components).
confidence (precision), support, coverage and lift are the
rule's Train numbers — measured on the rows it was found on.
precision_holdout and holds_up are its Test side: its precision
on held-out rows and the verdict on them (see
:func:parleml_mli.rules.scoring.holds_up).