Imbalanced data & outliers¶

Skewed targets — fraud, churn, rare-event detection — are the common case, not the exception.

In short

On a plain run you often need nothing else: pick a PR-aware eval_metric (average_precision, f1, beta) and let the tuned decision threshold work.
There is no automatic class weighting by default (engine.sample_weight = None). "balance_weight" turns it on; a column name gives per-row business weights.
Per-fold resampling (balancing.enabled) is off by default. When on, it reaches its ratio by deleting majority rows, so read the Rebalance: log line.
With hpo.provider="ml4_optuna" the imbalance treatment becomes one searched choice per model, scored on your metric.
Outlier removal (outliers.method) is a separate opt-in step on training rows only.

ml4 treats imbalance in three stages, each acting on a different part of the fit:

Stage	Acts on	Where	Gate
1 — resampling	the row set	per bag fold, inside the model	`balancing.enabled` (off by default; when on, guarded by the class ratio)
2 — imbalance-aware feature selection	the feature set	once, at plan time	always on
3 — the searched imbalance strategy	the objective	per roster model, in the tuner	`hpo.provider="ml4_optuna"`

The numbering is by depth, not by clock: stage 2 runs first, before AutoGluon sees the data at all.

Underneath all three sits the default path, which is what you get on a plain run.

The default path — the tuned threshold and your metric¶

You often need nothing else. ml4's post-fit decision layer tunes the threshold from the run's eval_metric: a single binary threshold, or per-class multiclass thresholds under a hard-label metric such as f1_macro. That alone recovers most of the achievable minority-class recall, without touching the training distribution at all. 0.5 is never an operating point ml4 assumes (a sweep may legitimately land on it).

Multiclass differs under a probability metric. With logloss — the multiclass default — per-class bars have no decision rule to win on, so none are fitted and predict serves plain argmax on the served probabilities, the Bayes rule for them. If a rare class needs its own bar, name a label metric such as f1_macro. The bars are then fitted and kept unless they score below argmax under that metric.

The threshold follows your metric's family, not the metric itself

There are three sweeps, and every metric maps onto one of them:

the F-β family (beta / beta_macro / beta_weighted) tunes F-β at beta_threshold (a top-level Ml4Config field, not an engine one);
accuracy and recall_weighted tune exact accuracy;
everything else — including roc_auc, logloss, average_precision and the macro/weighted precision & recall variants — tunes F1, because predict still owes you labels even when the metric never asks for any.

So a binary run graded on roc_auc ships an F1-optimal operating point; a multiclass run under a probability metric fits no bars and serves argmax (above). measures.json records which sweep ran as threshold_objective, next to the cutoffs themselves, so the substitution is visible rather than implied.

There is no automatic class weighting on the default path

engine.sample_weight defaults to None, so a classification run trains on the class prior of your data. cfg.engine.sample_weight = "balance_weight" turns class weighting on in one line, and stage 3 can measure the whole axis for you on your data.

Details: why class weighting is not the default

Two measurements argue against weighting every run:

It is a per-dataset bet, not a constant. Over 12 seeds × 5-fold OOF on three imbalanced fixtures (ml4/scripts/race_weight_vs_resample.py), weighting roughly doubles raw Brier on every fixture, while its ranking effect changes sign between them (−0.014 to +0.008 OOF AUC).
It would disarm stage 3. The searched none branch is scored by fitting the fold-local model directly with no weight vector, then deployed inside AutoGluon. A default global weight vector would hand every deployed model the weights anyway, so none would be measured unweighted and deployed weighted, and balanced_sample_weight would collapse onto it entirely.

Two roster families ignore engine.sample_weight entirely

engine.sample_weight is a TabularPredictor constructor kwarg, handed to every model, so it reads as a property of the run. For NN_TORCH and FASTAI it is not: AutoGluon passes them the vector and both drop it. ml4 warns rather than refuses, naming the offending members, and the roster panel badges them. The remedy that reaches every model is stage 1, which resamples the fold's rows.

Details: what weight-blind members mean for a run

AutoGluon logs the dropped weights below ml4's default verbosity, so nothing reaches the operator on its own. The opt-in foundation challenger is weight-blind for the same reason (none of the four wrappers references sample_weight).

This applies to a weight column exactly as much as to "balance_weight".

Nothing is wrong per model: each is honestly fit and honestly scored, and selection stays fair because the OOF metric is unweighted for everyone. What the run silently becomes is heterogeneous: some members corrected the prior and some did not, and the greedy ensemble blends the two.

balancing.enabled resamples the fold's rows, so the class balance every model sees really changes. The single source for the list of weight-blind families is roster.WEIGHT_BLIND_FAMILIES.

The most impactful lever for imbalance is therefore your metric, not a sampler:

from parleml_ml4 import Ml4Config, Tasks
from parleml_ml4.contracts.metrics import EvalMetric

cfg = Ml4Config()
cfg.ml_task = Tasks.BINARY_CLASSIFICATION
cfg.data.target_column = "is_fraud"
cfg.eval_metric = EvalMetric.average_precision   # or f1, or beta

average_precision — area under the precision–recall curve; the strongest default when positives are very rare.
f1 — balanced precision / recall.
beta with cfg.beta_threshold — an F-β that weights recall over precision (or the reverse) explicitly.

ml4 builds a real AutoGluon Scorer for the F-β family and user-defined metrics, so training and model selection optimise the metric you asked for, and the baseline honesty gate scores against the same one. See Algorithms & metrics.

Weighted training moves the raw probabilities

Once you turn weighting on, class weighting — with or without resampling — roughly doubles raw Brier versus an unweighted fit: the model is calibrated to a reweighted world, not yours. ml4's post-hoc calibration repairs it (calibration, on by default and gated on an out-of-fold bake-off — see Probability calibration). If you set calibration.apply_at_predict_time = False, expect distorted probabilities and judge the run on ranking metrics (average_precision, AUC) rather than Brier or log-loss.

With the search on, `balance_weight` is refused — a weight column is not¶

When you select hpo.provider="ml4_optuna", the class-weighting field under Engine stays available. One of its values conflicts with the search; the other two do not.

`engine.sample_weight`	With `hpo.provider="ml4_optuna"`
`None` (default)	The search owns the imbalance decision outright.
`"balance_weight"`	Refused. It is a `TabularPredictor` constructor kwarg, fixed for the whole run and applied to every model, so a per-model search cannot vary it — the searched axis collapses to one arm and measures nothing.
a column name	Legal. A weight column says how much each row matters, not how much each class does. It is a business quantity, and the search still chooses the class-prior treatment on top of it.

The template editor greys out exactly balance_weight, with the reason attached, generated from the same rule table the engine validates against.

The picker does not offer auto_weight at all. AutoGluon never implemented it (its constructor silently rewrites the value to None), so ml4 refuses it rather than let a run believe it is weighted when it is not.

Your own per-row weights¶

engine.sample_weight also accepts a column name, and that is a different thing from class weighting: it says how much each row matters (exposure, revenue, sampling probability), not how much each class does. The column must be numeric, non-null and non-negative; AutoGluon validates all three.

cfg.engine.sample_weight = "exposure"

One rule beyond that, and ml4 refuses the run rather than half-honouring it: the column must be present in the training frame and declared as neither a categorical nor a numeric feature.

Details: why a weight column cannot also be a feature, and other refused values

AutoGluon lifts the weight column out of the frame before any feature engineering runs and logs that it is "used as sample weights instead of predictive features". A column declared as a feature as well becomes a phantom predictor. It consumes a feature-selection slot (and can evict a genuine correlated feature), it is mined into interaction pairs that can only materialise as all-NaN, and it is planned into the GLOBAL recipe, which then indexes a column AutoGluon has already removed.

Two more values are refused up front rather than deep inside a fit:

"auto_weight": AutoGluon accepts it in its validator and then raises NotImplementedError from its learner, because the strategy was never implemented.
weight_evaluation = True with no weighting set: AutoGluon reads that flag only when a weight vector exists.

Passing a weight column from a host application¶

A weight column occupies a role no other column has: in the frame, but not a feature. An application that derives ml4's config by classifying columns — feature versus reserved — has no bucket for it. Concretely, the caller must:

keep the column in the DataFrame handed to fit;
leave it out of data.cat_features and data.num_features;
leave it out of whatever set the host uses to drop non-modelling columns — the name is read out of the frame by AutoGluon at fit time, so it must survive that drop;
set engine.sample_weight to the column name.

The library entry point satisfies all four by construction, since you build the frame and the config together. A UI that only offers "feature" and "ignore" cannot.

Details: the two failures, and how to tell them apart

Both of the buckets a host does have are wrong, and the failure is loud in each direction. The two messages are how you tell which one you hit:

What the host did	What happens	Message
Bucketed the column as a `cat_features` / `num_features` entry, because it is not the target, a key or an ignored column	ml4's config translator refuses the fit	`ValueError` from `engine.kwargs`, naming the column and the exact fix (remove it from the feature list)
Marked the column ignored so it would not be modelled, and dropped it from the frame	AutoGluon cannot find the weight column it was told to extract	AutoGluon's missing-column error at fit time

A weight column weights a regression fit too

"balance_weight" alone is classification-only. A column name is forwarded to the estimator on regression as well, so LGBMRegressor / Ridge / CatBoostRegressor all receive it.

Details: why a weight column reaches regression

AutoGluon's set_predefined_weights returns the frame untouched unless the problem type is binary or multiclass. A column name is extracted from the frame unconditionally (extract_column, before the problem type is consulted). Read "no effect on regression" as a statement about the predefined strategy only. The two symmetric mistakes are an analyst with exposure weights who leaves the field unset on a regression problem and silently loses the weighting they needed, and one who sets it and never reconciles the weighted fit with the unweighted RMSE the report shows.

Only the model is weighted — every layer around it is not

A weight vector reaches AutoGluon's estimators, and (with engine.weight_evaluation = True) AutoGluon's own validation score, ensemble blend and leaderboard. It reaches nothing ml4 owns. Probability calibration, the binary / multiclass decision thresholds, the constant-baseline gate, the conformal residuals behind prediction intervals and every number in measures.json's test block are all computed unweighted. measures.json → weighting states this on every run.

Details: what the unweighted layers mean, and the weighting block

A weighted run ships a model selected under the weighted metric with a threshold tuned, a calibrator fit and a baseline gate scored under the unweighted one. None of those layers takes a weights argument. On regression the gate compares against an unweighted training mean, which is the wrong constant for a weighted objective. With baseline.fail_on_no_signal set, that comparison can abort a run that is fine on its own terms.

Every run — weighted or not — carries a weighting block in measures.json:

"weighting": {
  "sample_weight": "exposure",
  "weighted_training": true,
  "weight_evaluation": false,
  "unweighted_layers": ["calibration", "decision_thresholds",
                        "baseline_gate", "reported_scores"]
}

sample_weight is read off the fitted predictor, not off the config, so it is what actually trained. unweighted_layers is task-shaped and empty on an unweighted run, where there is no divergence to disclose.

Stage 1 — guarded per-fold resampling¶

Class rebalancing is off by default: a skewed target is handled by the tuned decision threshold and the post-hoc calibration. Resampling moves the training prior (at 1% positives the defaults delete about 90% of the majority rows). It helps ranking only on small, weak-signal data where row-counting regularisers such as LightGBM's min_data_in_leaf bind. When enabled it does nothing unless the smallest class falls below the configured fraction of the largest:

cfg.balancing.enabled = True              # opt in (default False)
cfg.balancing.max_oversample_ratio = 1.5  # cap any class's inflation at 1.5× original
cfg.balancing.min_acceptable_ratio = 0.15 # floor: smallest class ≥ 15% of the largest
cfg.balancing.oversampler  = "randomoversampler"
cfg.balancing.undersampler = "randomundersampler"

Beyond two classes it is a levelling rule, not a floor¶

Both knobs are named in binary language, and on a binary target they behave exactly as named. On a multiclass target the resampler derives one ceiling from the rarest class alone and cuts every class above it to that count. Raising min_acceptable_ratio toward 1.0 makes the run more aggressive, not gentler.

Details: the multiclass ceiling, worked through

The ceiling is

new_max ≈ rarest_count × max_oversample_ratio / min_acceptable_ratio   (clamped to the largest class)

Two folds of the same 3-class problem, at the shipped defaults (1.5 / 0.15):

fold (before)	ceiling	after	what happened
100 / 50 / 10	100	100 / 50 / 15	ceiling lands on the head count; nobody is cut
100 / 50 / 5	46	46 / 46 / 7	head more than halved, and level with the middle class

Dropping the rarest class from 10 rows to 5 — a change in a class that is not the head and not the middle — more than halves the head class. It also flattens a genuine 2:1 head-to-middle prior that was never imbalanced in the first place. The arithmetic is the only reading of two binary knobs that generalises to K > 2. On a multiclass target, read min_acceptable_ratio as "how much class-prior distortion I accept across the whole roster of classes", not as a floor under one of them. Raising it lowers the ceiling and deepens the trimming.

To leave the priors untouched, keep cfg.balancing.enabled off (the default) and reach for engine.sample_weight (the default path) instead. Whatever the run does, the per-class before/after counts are logged, so the effect is never a guess:

Rebalance: 155 -> 99 training rows in this fold (64% kept); per class A: 100 -> 46, B: 50 -> 46,
C: 5 -> 7. 58 row(s) were DELETED: with minority inflation capped at max_oversample_ratio=1.5x,
trimming the larger classes is the only way to reach min_acceptable_ratio=0.15.

The line is logged at WARNING (not INFO) whenever fewer than half the fold's rows survive.

Random sampling duplicates/removes complete rows and is categorical-safe. The sampler dropdowns also expose SMOTENC plus numeric-only SMOTE/ADASYN/Borderline/K-means/SVM and NearMiss/Cluster-Centroids methods.

Resampling runs inside the model's _fit: only the training partition in train/validation mode, and independently on each fold's training side in cross-validation mode. Fold-local encoders fit on the original training rows before outlier removal → oversample → undersample; synthetic methods operate on that already transformed training matrix. It applies to the training slice only, so out-of-fold predictions stay honest. This is a merge-blocking leak-safety invariant (OOF row count unchanged, noise stays at chance).

max_oversample_ratio is a multiplier, not a floor

It is a cap on minority inflation expressed as a multiple of the minority class's original size. A value below 1.0 shrinks the minority class rather than growing it, the opposite of what the knob's name suggests. BalancingConfig.__post_init__ rejects that outright, along with a min_acceptable_ratio outside (0, 1].

The ratio is met by deleting majority rows, and it has no floor

Read the two knobs together. Once the max_oversample_ratio cap binds, trimming the larger classes is the only arithmetic left. At the shipped 1.5 / 0.15, a binary fold's training slice becomes 11.5 × n_minority rows, independent of how much majority data exists. At 1% positives 11.5% of the rows survive; at 0.1%, 1.1%. Raise max_oversample_ratio or lower min_acceptable_ratio to keep more of the majority.

Details: the deletion arithmetic and the measured effect

min_acceptable_ratio is a floor relative to the largest class, and max_oversample_ratio caps how far the smallest class can be inflated. Do the substitution and the majority count cancels out:

new_max = min_size · max_oversample_ratio / min_acceptable_ratio

At 1.5 / 0.15 that is 10 × n_minority for every class. On ml4's own rare fixture (1,648 rows, 2.9% positives) that is 1648 → 552: +24 minority rows against −1,120 majority rows.

This is the only way to honour what you asked for, and ml4's 12-seed paired race measured it as a gain in exactly that regime (+0.0205 OOF AUC over weights alone at min_data_in_leaf=20). But it is not what "oversample the minority" sounds like, and there is deliberately no absolute row floor: clamping would quietly break the ratio you configured. So the run reports it instead — per fold, per class, at WARNING once less than half the slice survives:

Rebalance: 1648 -> 552 training rows in this fold (33% kept); per class
0: 1600 -> 480, 1: 48 -> 72. 1120 row(s) were DELETED: with minority inflation
capped at max_oversample_ratio=1.5x, trimming the larger classes is the only way
to reach min_acceptable_ratio=0.15.

Read it before trusting the preset and time budget you picked: both were sized for the full frame, and the row-counting regularisers bite against the smaller count.

Stage 1 composes with the weighting — it does not replace or double-correct it

Both act on the same quantity — how much each class counts — so the rebalance is mass-preserving: each class leaves it with exactly the total sample-weight mass it entered with. The weighting decides how much a class counts; the resampling decides only how many rows carry that mass. balancing.enabled leaves engine.sample_weight alone.

Details: what resampling adds on top of weights, and when ml4 warns

Your weight column keeps the class ratio you asked for instead of being multiplied by the oversampling factor.

ml4 warns about the combination only when engine.sample_weight = "balance_weight", balancing.enabled = True and calibration.apply_at_predict_time = False all hold: then the class-weighted raw probabilities serve uncorrected. Measured over 12 seeds × 5-fold OOF on three imbalanced fixtures (ml4/scripts/race_weight_vs_resample.py), keeping the weighting next to resampling is calibration-neutral once the post-hoc calibrator has run, and worth up to +0.007 OOF AUC where the row-count regularisers bind.

What resampling adds on top of weights is exactly that row count: it changes the row-counting regularisers (LightGBM's min_data_in_leaf and friends), which a weight cannot do. Its weight-side twin min_sum_hessian_in_leaf sees the same mass either way. The effect runs in both directions: extra minority rows relax them, and the majority rows the ratio deletes tighten them against a much smaller frame (see the danger box above — on a heavily skewed target deletion is the dominant effect, not inflation). The gain shows up only when those regularisers actually bind, which is why the knob is gated on the class ratio rather than resampling every run.

Why resampling is leak-safe here

Fold-local encoders are fit once on the true training distribution. Random methods reuse those leak-free rows; synthetic methods generate rows only in the resulting transformed training space. Validation, calibration, OOF and test rows are transformed normally and never passed to a sampler. It applies to every stock family (the boosters, RF, XT, linear and both neural nets), because all of them are wrapped in the same fold-local model. The regulated / glass-box models are left untouched — see the roster warning for what that means for a roster.regulated run. It is a no-op for regression.

The sampler safety matrix¶

Categorical safety. A sampler that computes distances between rows needs a metric space. Every neighbour-based sampler — the SMOTE family, the cleaning under-samplers, the hybrids — interpolates or measures in numeric space and breaks on categorical columns. Only three samplers in the catalogue are categorical-safe: randomoversampler and randomundersampler (they duplicate or drop whole rows, so they never have to compute anything), and smotenc (built for mixed frames — it takes the modal category rather than interpolating it).

Sampler	Categorical-safe	ml4 status
`randomoversampler`	✅	Accepted — the default oversampler
`randomundersampler`	✅	Accepted — the default undersampler
`smotenc`	✅	Accepted — categorical-aware synthetic rows
`smote`, `adasyn`, `borderlinesmote`, `kmeanssmote`, `svmsmote`	❌ numeric-only	Accepted when no categorical columns remain
`nearmiss`, `clustercentroids`	❌ numeric-only	Accepted when no categorical columns remain

The template editor detects active categorical model features (after ignored and reserved columns are removed) and disables incompatible choices. Enable Use numeric attributes only to remove remaining non-numeric columns after numeric encodings are created; the same columns are then omitted at validation and inference. Unsupported method names are rejected during fit setup rather than silently substituted.

Stage 2 — imbalance-aware feature selection¶

This stage is always on and needs no configuration. It decides what the later stages get to work with.

ml4's plan-time instrumental models — the feature-selection rankers and the tree-guided interaction miner — are themselves fit with balanced inverse-frequency sample weights (minority/majority ratio n_majority / n_minority, keyed on class frequency rather than on the label 1, so it is coding-agnostic). Both rankers are weighted, not only one.

Without this, a ranker fit on a 2%-positive frame ranks features by how well they explain the majority, and a genuinely minority-predictive column can be pruned before AutoGluon ever sees it. No amount of downstream reweighting can undo that error.

There is deliberately no weight floor here

A max(50, …)-style override would apply a ~50× minority weight to near-balanced data and distort every data-driven structure decision — feature selection, correlation pruning, interaction-pair ranking — on the default path, for every run. The instrumental weighting is the plain balanced ratio and nothing else. The data-adaptive ceiling that does exist belongs to stage 3, where it is searched rather than assumed.

Stage 3 — the searched imbalance strategy¶

When hyperparameter search is on, the imbalance treatment stops being a constant you guessed once and becomes one searched categorical per family, scored on the run's metric like every other hyperparameter.

cfg.hpo.provider = "ml4_optuna"  # stage 3 rides on the tuner

Why one axis and not several knobs¶

Sample weights, a native class weight and resampling all correct the same quantity: the class prior. Stacking two of them corrects it twice, which overshoots into the minority class and wrecks the probability scale on the way. Independent knobs would put that mistake one trial away, and a search will find it if it happens to score well on the single inner slice. Modelling them as alternatives makes double correction impossible by construction. Exactly one mechanism is active per trial.

Resampling is the one of the three the search does not own: it is fixed for the run by the stage-1 ResampleSpec and reaches the fold-local models whole, so a per-trial search cannot vary it. It is still exclusive against the two branches below; config.dependencies enforces that exclusion, not the axis.

Branch	What it does	Tasks
`none`	Train on the distribution as it is. Kept in the candidate set so the search can decide the treatment is not worth it — which on well-separated data it sometimes is.	all
`balanced_sample_weight`	Per-fold inverse-frequency row weights, computed inside the fold-local model.	classification
`native_class_weight`	The family's own knob — see the table below.	where the family has one

On regression the whole axis collapses to ("none",) and is not sampled at all. Every branch re-weights a class prior, so offering the choice would burn a search dimension on a decision that cannot change the fit.

The native knobs, per family¶

The strategy is searched per roster model — every booster variant runs its own study — but the knob is a property of the library. So this table is keyed by family, and every variant of a family searches the same one (roster.family_of is the projection).

Family	Knob	Searched values	Tasks
`GBM`	`scale_pos_weight`	`[min(1, n_neg/n_pos), max(50, n_neg/n_pos)]`, log	binary only
`XGB`	`scale_pos_weight`	same	binary only
`CAT`	`auto_class_weights`	`Balanced` \| `SqrtBalanced` \| `None`	binary and multiclass
`RF`, `XT`	`class_weight`	`balanced_subsample` \| `None`	binary and multiclass
`LR`, `glm`, `scorecard`	`class_weight`	`balanced` \| `None`	binary and multiclass

Details: four load-bearing details in that table

Both ends of the scale_pos_weight range are data-adaptive, and they are mirror images: max(50, n_neg/n_pos) and min(1, n_neg/n_pos). A hard ceiling of 50 puts the textbook optimum out of reach on genuinely heavy imbalance (at 99:1 the balancing value is 99). A purely ratio-based ceiling would collapse to a useless range on near-balanced data; taking the larger keeps both searchable. The floor mirrors that. A constant floor of 1.0 would assume the positive class is the rare one. It need not be: converted, approved and on_time are ordinary majority-positive targets. On those the balancing value is n_neg/n_pos < 1, and a constant floor would make it unreachable, so every drawable value would push further in the direction that already dominates. Together the two bounds guarantee the interval contains both landmarks that matter: 1.0 (no correction) and the balancing ratio. The scale is log, because the interval spans decades under heavy imbalance and 1.0 sits inside it rather than at an end.
n_pos / n_neg are the counts of internal class 1 / class 0, not of the minority and the majority. scale_pos_weight scales class 1, and class 1 is whichever label your positive_class names. The tuner hands it to AutoGluon's label cleaner so the search and the deployed fit code the labels identically (see the positive class is part of the label space). Reading the two counts off the frequency ranks instead would hard-code "positive = minority" and detach the searched range from the class the parameter actually weights.
scale_pos_weight is binary-only. LightGBM and XGBoost ignore it under multiclass softmax. So on a multiclass run those two families are not offered native_class_weight at all. balanced_sample_weight covers K > 2 for them, and CatBoost's auto_class_weights covers it natively. No branch quietly does nothing.
balanced_subsample, not balanced, for the forests. It recomputes the weights per bootstrap sample, which is the correct variant for a bagged learner; plain balanced uses the whole training set's frequencies for every tree.

engine.sample_weight is a different locus — leave it off while the search owns the axis

engine.sample_weight is a TabularPredictor constructor argument: one global decision for the whole run, which a per-model search cannot vary. The searched balanced_sample_weight is computed per fold, inside the model. They are different loci, and only one of them is a decision the search can make. Leaving engine.sample_weight at its None default is what lets the search measure the decision instead of inheriting it.

Details: how the tuner handles a run-level weight

Two things follow, and both are enforced rather than left to the reader:

Every trial is fitted under the deployed weighting. The tuner builds its models outside TabularPredictor, so it reproduces AutoGluon's weight vector itself: the balance_weight formula n / (count_c · k), or your weight column normalised the way AutoGluon normalises it, with that column lifted out of the features exactly as AutoGluon lifts it. Otherwise the trials would optimise a loss the winner never trains on: not only the imbalance branch but every capacity knob. XGBoost's min_child_weight — a sum of hessians — binds n_neg/n_pos times harder once minority rows carry weight.
Branches that cannot change the fit are removed from the axis. With engine.sample_weight="balance_weight" the class prior is already corrected, so the axis collapses to ("none",) and the run says so in the log. With a weight column, balanced_sample_weight is dropped (one sample_weight slot per fit, and Ml4RecipeFoldLocalModel._fit refuses to overwrite AutoGluon's vector), while native_class_weight stays, because a business weight is not a class-prior correction.

Outlier removal (orthogonal, opt-in)¶

An independent step with its own gate on cfg.outliers: drop noisy training rows with an unsupervised detector before the rebalance.

cfg.outliers.method = "iforest"           # None (default) | "iforest" | "ee" | "lof"
cfg.outliers.threshold = 0.05             # contamination fraction, in (0, 0.5)
cfg.outliers.columns = "original_numeric" # or "all_numeric"
cfg.outliers.min_samples = 100

Knob	Meaning
`method`	The detector — `iforest` (IsolationForest, most general), `ee` (EllipticEnvelope, Gaussian-ish features), `lof` (LocalOutlierFactor, local density, slowest). `None` disables the step.
`threshold`	Expected outlier fraction (sklearn `contamination`). Keep `< 0.5`.
`columns`	`original_numeric` (default, safer — only user-declared `num_features`) or `all_numeric` (every post-encoding numeric, including target-encoded columns; more aggressive).
`min_samples`	Floor below which the detector is skipped rather than fit on a handful of rows.

Behaviour is implicit from ml_task:

Classification → the detector is fit per class, so the minority class is never deleted on aggregate. Classes below min_samples are skipped entirely, which protects tiny minority classes by construction.
Regression → a single-pass fit on the whole training set, skipped below min_samples rows.

Outlier removal touches training data only — validation and test frames are left intact — and never passes raw categorical columns to the detector. Its gate (outliers.method is not None) is independent of balancing.enabled, but not of the roster: see below.

A regulated roster ignores this whole page's Stage 1 and outlier removal

Per-fold rebalancing and outlier removal run inside the eight stock families' fit, and the regulated roster (glm, rulefit, scorecard, ebm) deliberately never carries them: a transparent model trains on the distribution the auditor was shown. With roster.regulated = True every balancing knob and all four outliers knobs are inert, on every task. The template editor still renders both sections, because its only visibility gate is the ML task. ml4 logs this at config validation; it does not switch the knobs off, so moving back to a stock roster restores your choice.

The remaining lever on a regulated run is engine.sample_weight, which does reach glm / scorecard / ebm. RuleFit takes neither: imodels' implementation has no sample_weight parameter, so a regulated RuleFit run on an imbalanced target gets its imbalance handling from the tuned decision threshold alone.

What to reach for, when¶

Situation	Do this
First run on a skewed target	Nothing. Set a PR-aware `eval_metric` and let the tuned decision threshold work.
Recall still too low, positives rare (≲ 3 %) and the data small	Try `engine.sample_weight = "balance_weight"`, or opt into stage 1 with `balancing.enabled = True` — measured on a 2.9 %-positive fixture, resampling is worth +0.027 OOF AUC over weighting alone. They are not either/or (they compose), but weighting is the one that costs raw probability accuracy.
Recall still too low, classes already well separated	Leave `balancing.enabled` off (the default). On a strongly-separable fixture resampling cost 0.006 AUC (p = 0.02) — there is no leaf the extra rows unlock. Invest in features and the threshold instead.
You have the budget and want the decision measured rather than guessed	Select `hpo.provider="ml4_optuna"` and leave `engine.sample_weight` at its `None` default, so stage 3 searches the axis on your data and your metric.
Multiclass with a rare class	The searched axis covers it: `balanced_sample_weight` for LightGBM/XGBoost, `auto_class_weights` for CatBoost, `class_weight` for the forests and linear models. `scale_pos_weight` does not apply. For per-class decision bars, name a label metric such as `f1_macro`; under the default `logloss`, `predict` is plain argmax.
A few extreme rows dominate the boosters	Add `outliers.method = "iforest"`.
You specifically need SMOTE	Select a SMOTE-family method; use SMOTENC for mixed data or enable numeric-only attributes.