Skip to content

Experiment tree

In short

  • The tree shows a workspace's experiments as a family: an arrow runs from the experiment a run was derived from (its parent) to the new one.
  • Each arrow names the levers that changed: data, recipe, algorithm, features, split, metric or parameters.
  • Scores are selection scores, and a change against the parent shows only when the two are comparable. Positive is always better.
  • Open it from a workspace's Experiments page (List | Tree → Tree).

The experiment tree shows a workspace's experiments as a family tree: each experiment is a node, and an arrow runs from the experiment it was derived from (its parent) to the new one (its child). It answers "what did we try, what changed each time, and did it help?" without opening every run.

Open it from a workspace's Experiments page (List | Tree → Tree) or the Overview's Experiments lens. The same data is GET /api/workspaces/{workspace_id}/experiment-tree.

Nodes

A node shows the experiment's name, status, score and its change against its parent. Two markers pick out the experiments that matter:

  • In production / In shadow — the registry's fact for the model registered from this experiment (the experiment page also shows the other stages: "Registered · Validated"). A model in production imported from outside has no experiment, and the tree says so.
  • Group best — the best selection score among experiments that can be compared (below), marked only when at least two succeeded experiments share that comparison; one per group, a tie going to the earliest (an iteration that only matches its parent has not beaten it).

The score is the selection score — the one experiments are chosen on — and the card names its split: val (a supervised, recommendation or ranking run's validation split), CV (a forecast's cross-validation selection windows) or in-sample (clustering, anomaly and association runs, which have no split). Group best and every change against a parent use it. The held-out test score is shown beside it to read once for the experiment you pick, never to rank: choosing the best run on test would make test a selection score and bias it. A run with no selection score is not ranked. Experiments whose parent was deleted show as roots, with a note.

Clicking a node opens its panel: the levers and the settings that changed, the features added and removed, the hypothesis, Open (the experiment page) and, when an agent session launched it, a link to that moment in the session.

Levers: what changed on an edge

Each arrow is labelled with the levers that changed between parent and child, biggest change first:

Lever What it covers Example label
data the dataset or the target data: churn_2025 → churn_2026
recipe the recipe that built the dataset, or a new version of it recipe: new version
algorithm the model roster or learner LightGBM → LightGBM + CatBoost
features the fit-time feature set, else the feature-engineering settings +3 / −1 features
split the validation strategy and split settings split: random → temporal
metric the optimisation or evaluation metric metric: roc_auc → log_loss
parameters every other setting learning_rate: 0.1 → 0.05

Features are counted exactly when both runs recorded the feature set they were fitted on; the panel then lists the added and removed columns. Otherwise the feature-engineering settings that changed stand in. An edge with no lever reads no settings changed, or copy for a duplicate (drawn dashed). The panel shows the full settings diff (the first 60 changes).

The diff is against the parent's settings as they were when the child was created: the child keeps a snapshot of them. If the parent is rerun with other settings later, the edge does not change, and the panel says the parent changed since.

Score deltas: only between comparable experiments

A child's "+0.012 vs parent" appears only when the two are comparable: same experiment type, target, metric, selection-score split and recorded selection design, and a metric whose direction is known. Forecast design includes frequency, resampling, horizon, CV window count, step size and gap; changing these makes the scores incomparable. Deltas use selection evidence, not the held-out test score. The delta is signed so that positive is always better, whether the metric is higher-is-better (AUC) or lower-is-better (log loss), and the edge is coloured green or red to match. Anything else reads Not comparable with its parent — a different target or metric is a different question, not a worse answer.

The grouping is a recorded comparison rule, not a scientific guarantee: outside forecasting, it does not establish identical datasets or identical validation rows. Check the data and validation design before interpreting a group's winner as an improvement. A parent re-baselined by a later constraint hypothesis is excluded from Group best.

Where a parent comes from

An experiment records its parent, and how it came to exist, when it is created. The source is one of:

Source Made by
plan launching a plan — by a person or an agent through run_experiment. A plan names its parent (run_experiment(action="plan", parent_experiment_id=…)) and may carry a hypothesis: the lever and the expected effect, shown on the edge.
clone Edit settings on an experiment, in the wizard.
replicate Duplicate: a finished copy of the run.

An experiment made any other way (a new wizard run, an adopted external run) is a root. A parent in another workspace is refused; a parent deleted before the launch leaves the child a root.

When an agent session launched the experiment, the lineage also names the session, the step and the skill that ran it, from the provenance ParleML recorded when it gave that step its token.

Lineage from stored plans

Details: backfilled lineage

An experiment launched from a plan but with no recorded lineage gets its parent and hypothesis from the stored plan file when the catalog is rebuilt, and carries backfilled: true. The backfill never overwrites lineage an experiment already has, and a parent that is gone or in another workspace leaves the child a root.

Sessions and their experiments

Every experiment an agent session launched names that session in its lineage, so the tree knows which experiments each session produced. From a session, Show its experiments opens the tree with those experiments highlighted; from a node launched by a session, the panel's by · session link goes back to the step that launched it. The experiment page shows the same lines: "Derived from parent name (levers)" and "Hypothesis · lever: “expected effect”", the latter linking to that step (a root experiment a plan launched, such as the baseline, shows its hypothesis too). The page refreshes the tree when it sees its run end, so its Best so far and Registered · In production chips are current.

Next