compare

Scope: read · Signature: compare(kind: {experiments|models|datasets}, ids: list[str], include_production_model: bool = False, reference_id: str | None = None, feature: str | None = None) -> Any

Use to compare like with like — and to answer "how does it compare with production?".

kind="experiments": 1–4 experiment ids of one workspace; include_production_model adds the experiment of the workspace's model in production (a model in shadow: pass its experiment id, from the brief). Returns aligned scores, per-split metrics with the best marked, trust grade, top drivers, and the resolved-config settings that differ. Split metrics, trust and drivers read the explanation profile: unavailable names a run without one yet — wait for its explain stage (the run_all chain), or run_experiment(action="explain") a run launched without it. kind="models": 2–4 registered model ids → lineage and headline scores. kind="datasets" (compute token): drift of 1–3 datasets (ids) against reference_id — PSI per feature with tiers, highest drift first; with feature= and one id, that feature's bins side by side. It computes but changes nothing, so the tool stays read-only; an analyst token is refused this kind. Never recompute these numbers yourself.

Powered by: the backend comparison service (backend/app/services/experiment_compare.py).