Skip to content

Split policy

SplitPolicy declares how the one training frame is partitioned for validation and the outer test set — IID/stratified, grouped, or time-aware — and is resolved by the split resolver and the fit's validation helpers. AutoSupervised.fit handles inner validation; ParleML Studio or a library caller using resolve_split carves the outer test before fitting. See the data-splitting guide.

In short

  • Set the policy on engine.split_policy; it defaults to iid_stratified with 5 folds and a 20% outer test (outer_test_frac=0.2).
  • grouped keeps every row of one group_column value on one side of each split.
  • temporal_holdout needs a time_column and trains on the past, validating on the future.
  • The policy has its own seed (default 0), separate from execution.seed.

SplitPolicy dataclass 

SplitPolicy(
    strategy: SplitStrategy = IID_STRATIFIED,
    group_column: str | None = None,
    time_column: str | None = None,
    n_folds: int = 5,
    embargo: str | None = None,
    outer_test_policy: OuterTestPolicy = RANDOM,
    outer_test_frac: float = 0.2,
    validation_cutoff: str | None = None,
    outer_test_cutoff: str | None = None,
    allow_multilayer_stacking: bool = True,
    seed: int = 0,
)

Declared fold geometry (§3.5.2).

Field roles: group_column is the group-carve key, time_column the event-time axis, and n_folds the inner CV fold budget.

isolation_columns property 

isolation_columns: tuple[str | None, str | None]

(group_column, time_column) that actually isolate this policy's folds.

The strategy decides, not which columns happen to be filled in — the same rule the outer split follows (fe.split_resolver._inner_folds dispatches on strategy), so one concept governs both boundaries instead of two rules that can disagree.

They did disagree. A GROUPED policy may also name a time_column (nothing forbids it, and the repo's own fixtures do), and the fold-local out-of-fold encoders resolved their inner split from whichever key arrays were non-empty (fe.transformers._oof.build_oof_folds tests time_order first). So the outer folds isolated customers while the inner ones isolated time — the same customer on both sides of every inner boundary, which is the leak the group column was declared to prevent, and no disclosure because nothing had failed.

Anything that needs "the key this run isolates on" reads it here: the planner (which stamps it into the recipe steps), :func:~parleml_ml4.engine.fit_geometry.isolation_key_frame (which carries it to the fold-local chain) and the hyperparameter search's fold carve.

SplitStrategy

Bases: str, Enum

Next: Data splitting guide · Ml4Config & DataConfig