Deployment & export

A fitted ml4 model is served as a FittedPipeline. This page covers the three ways to take it to production: the directory bundle (full fidelity), the Treelite accelerator (faster tree core), and the airgapped export (no AutoGluon at serve time). For the prediction surface itself see Inference & fast scoring; for MLflow / Unity Catalog see MLOps.

In short

• Save the full-fidelity bundle with model.save(path) and reload it with parleml_ml4.load_pipeline(path). The bundle is a directory, and a bundle of another major format is refused.
• engine.ensemble_advantage_margin (default 0.005) ships AutoGluon's weighted ensemble only when it beats the best single model by more than 0.5%.
• A model with pseudonymised identifier columns needs ML4_PSEUDONYMISATION_KEY set wherever it serves.
• The Treelite accelerator speeds up a single LightGBM or XGBoost booster; the airgapped export serves a single GLM, scorecard, RuleFit, LightGBM or CatBoost model without AutoGluon.

The ensemble advantage margin

Everything below depends on what the run decided to ship. AutoGluon's greedy weighted ensemble becomes the strongest model whenever it beats the best standalone model by any epsilon on out-of-fold data, including an epsilon that is pure validation noise. ml4 then pays for that epsilon three times over:

• inference latency: every seated member is scored on every row;
• interpretability: a multi-leaf strongest model degrades what the MLI layer can say about it, because there is no single booster to read native importances and tree SHAP off;
• deployability: both fast paths on this page require a single model. The Treelite accelerator needs exactly one LightGBM or XGBoost booster, and the airgapped export refuses an ensemble outright.

engine.ensemble_advantage_margin states the price the ensemble has to earn. It defaults to 0.005: the guard is on, and an ensemble must beat the best single model by more than 0.5% of that model's score to ship:

cfg.engine.ensemble_advantage_margin = 0.01   # must beat the best single by MORE than 1%
cfg.engine.ensemble_advantage_margin = 0.0    # opt out: AutoGluon's choice stands

0.5% is roughly the run-to-run noise of a cross-validated score, so a narrower "win" than that is not reliably a win at all, while the ensemble's costs are certain.

The test is relative: the improvement over the best standalone model, as a fraction of that model's own score:

(ensemble_score - best_single_score) / max(|best_single_score|, 1e-12)  >  margin

Relative rather than absolute because an absolute delta means nothing without the metric: 0.01 is a rounding error on an RMSE over a $1M target and a landslide on an AUC of 0.99. The comparison runs before the post-fit layers are fitted, so calibration, thresholds and intervals are all built on the model that ships.

Value	Effect
0.0	The guard is off: AutoGluon's choice stands, whatever the epsilon.
0.005 (default)	The ensemble must exceed the best single by 0.5% of that model's score. Meeting the margin exactly is not exceeding it, so it downgrades.
Up to (not including) 1.0	Progressively stricter. 1.0 and above are rejected: demanding twice the best single model's score is unreachable for a blend of those same learners, so it would be a disguised "never ensemble" switch. Say that with a single-model preset, or by setting engine.training_policy = "train_validation", which never trains the blend at all.

Details: four things to know before you change it

• The commonest thing it removes is the ensemble of one. AutoGluon's greedy selector routinely puts all of the weight on a single base model and still ships a WeightedEnsemble_L2 wrapper around it. That wrapper predicts exactly what its one member predicts, so its score is identical and its relative gain is 0.0, below any margin. The downgrade there is free: the same predictions, one layer of indirection less, and a strongest model the accelerator and the exporter can read.
• The alternative is the best of every standalone model, seated members and candidates AutoGluon rejected alike. Comparing only against seated members would measure the ensemble against its own worst-case alternative and let a narrow blend win by construction.
• A downgrade keeps the deploy shape. If the run is serving refit-on-all-data models, the single model's refit_full twin ships; the downgrade does not quietly undo engine.refit_full.
• No evidence, no downgrade. If a score is missing on either side, if no standalone candidate carries a score, or if the chosen model is not servable (for example a single-model run's keep_only_best already deleted it), the gate records why and leaves AutoGluon's strongest model alone. The deployed model is only ever swapped on measurement.

Reading the verdict

The decision is written beside the predictor as ml4_ensemble_gate.json, inside the predictor directory, so it travels into the deployed bundle. It is republished in the ensemble composition sidecar as an ensemble_gate block carrying both scores, the relative gain, the margin it was tested against, and a reason from cleared_margin / below_margin / guard_disabled / no_single_candidate / unscored / not_deployable. The run log says the same thing under ENSEMBLE_GATE_DOWNGRADE / ENSEMBLE_GATE_OK.

The record has to exist because the number does not survive the swap: once the strongest model is re-pointed at the single model, the losing ensemble's score is not reachable from the board the sidecar is composed from.

margin_downgraded_to_single means a real downgrade

The sidecar's margin_downgraded_to_single flag is set only by a real downgrade, with the arithmetic attached. "An ensemble was trained and a single model is the strongest" is not enough, because that is also exactly what AutoGluon picking a single model on merit looks like.

Pick the margin from the seed spread, not from taste

The right margin is the size of the run-to-run noise on your data. Measure it: refit at several execution.seed values and take the spread of the score (see Reproducibility & seeds). A lift_vs_single smaller than that spread is not an advantage; it is the ensemble winning a coin flip that the sidecar then invites you to read as a real win.

Details: the two sides of lift_vs_single are not measured the same way

AutoGluon scores its greedy weighted ensemble on the rows the blend weights were hill-climbed on. On the bagged branch it fills the combiner's "OOF" by predicting on the training matrix outright (can_get_oof_from_train). best_single_score is a genuinely held-out number. So the subtraction pits an optimistic value against an honest one, and the margin is doing two jobs at once: absorbing seed noise and crudely discounting the combiner's selection optimism.

Every artifact says so rather than leaving it to be inferred. The composition sidecar carries ensemble_score_in_sample, each leaderboard.json row carries score_is_in_sample (so a board ranking the combiner among honestly-scored base learners is legible as the asymmetric comparison it is), and the gate record carries the same flag. The Ensemble tab reads them: a positive lift on an in-sample ensemble score is shown as optimistic, not as a win.

The margin does not stop the ensemble being trained

It is a shipping policy applied after the fit, so the ensemble is still built and still appears on the leaderboard with its score, which is what makes the downgrade auditable. To skip building it altogether, pick a single-model or regulated preset, or set engine.training_policy to a non-cross-validation policy yourself: the weighted ensemble is part of the run's fit shape (parleml_ml4.config.resolve_fit_shape), built on every cross-validation run and on no other, so there is no separate switch for it.

Saving and loading the bundle

FittedPipeline.save(path) writes a directory bundle, not a single pickle: the served model is an AutoGluon predictor directory plus ml4's post-fit layers.

import parleml_ml4

model.save("out/bundle")                      # writes a directory
reloaded = parleml_ml4.load_pipeline("out/bundle")    # FittedPipeline.load under the hood
reloaded.predict_proba(X)

The bundle layout:

out/bundle/
  manifest.json     # format_version + ml4 version + strongest model + task (validated on load)
  predictor/        # the AutoGluon predictor directory (both FE loci)
  layers.pkl        # post-fit layers (calibration, thresholds, interval, ...), LZMA-compressed
  recipe.json       # human-readable FE plan — audit sidecar (not read on load)
  measures.json     # fit-time measures — audit sidecar (not read on load)
  decision_basis/   # what the decision layer was fitted FROM (optional)

layers.pkl is the source of truth on load; recipe.json and measures.json are human-readable audit sidecars and are never read back.

decision_basis/ carries the strongest model's out-of-sample probabilities, the labels they were graded against, the bake-off's fold geometry and the training target's class histogram: everything calibration, the decision threshold and the baseline gate were fitted from. It is what makes those three re-decidable without retraining. It is its own directory rather than a layer inside layers.pkl because the frame does not compress meaningfully, and a reader that wants only the basis should not have to unpickle every layer to reach it. It is absent when deployment.persist_decision_basis is off (default on), or when the frame exceeds deployment.decision_basis_max_cells (default 50,000,000).

It contains labels, in reduced form

The basis holds evaluation labels as class codes and the training target as a class histogram, never the raw vectors, and never the group or time keys the folds were carved from. That is a deliberate reduction, not an absence: a bundle is still a derived-data artefact and should be shared on the same terms as measures.json.

load reconstructs the facade atomically and refuses a bundle a major format_version behind or ahead of the current ml4 (an incompatible layout). It raises the same ValueError every other versioned ml4 artefact raises, naming the on-disk format, the format this build reads, the ml4 that wrote the bundle, and the remedy (re-train the model). See Troubleshooting. The current bundle format is 5.

The measures.json beside it (and the run's metrics/measures.json) carries a measures_version. A reader accepts exactly the version this ml4 writes and refuses any other, a missing stamp included. A number whose meaning differs between versions (the Brier convention, the shape of the baseline_gate block) is therefore never read under the wrong meaning. The platform answers such a run's Scores and Decision-layer views with a 409 naming both versions; re-run the experiment to regenerate the file.

Save paths

AutoSupervised.save(path) requires a target: either pass save(path=...) or construct AutoSupervised(config, path=...). save returns the bundle path; mkdir(parents=True, exist_ok=True) means an existing directory is reused, so point at a fresh directory if you want isolation.

The pseudonymisation key

A model that declares card, IBAN, phone, email, name, IP or device-info columns replaces them with keyed pseudonyms (see Stateless encoders), and ships without the key: the bundle's recipe.json records only the key's fingerprint (pseudonymisation_key_fingerprint on each identifier step), and no pickle holds the key. Every environment that serves such a model must set ML4_PSEUDONYMISATION_KEY to the key the model was fitted with:

• Bundle (load_pipeline): loading does not need the key; the first predict / predict_proba raises PseudonymisationKeyError, naming the fingerprint of the key the model needs, when the variable is unset or holds another key.
• MLflow pyfunc: set the variable in the serving environment (the serving container's environment, or the endpoint's environment variables backed by a secret scope on Databricks). load_context checks it, so a host without the key fails when it loads the model rather than on its first request. The logged model stores no input_example rows that contain a pseudonymised column (the signature is still inferred from them), so no raw identifier is logged beside it.
• Airgapped export: the exported encoders carry only the fingerprint too, and read the variable at their first prediction in the same way.

To see which key a model needs, compare the fingerprint in its recipe.json with parleml_ml4.contracts.pseudonymisation.key_fingerprint(<key>). ParleML Studio exports its installation's key for its own process and workers; a separate serving host needs the same value set explicitly (see "Identifier pseudonymisation key" in the backend README). Never generate a new key for an existing model: its pseudonyms would match nothing it learned, which is why it refuses to serve under one.

Serving-latency budget

A hard per-row SLA is expressed as a ceiling on the served model's prediction cost:

cfg.deployment.max_inference_seconds_per_row = 0.002   # 2 ms/row
cfg.deployment.inference_latency_batch_size = 256      # the batch production sends

Models that miss the ceiling are dropped from the ensemble during the fit, so this is a real constraint on what gets trained, not a post-hoc filter. Both settings default to None (no budget).

Three things are worth knowing before setting it:

• It lives here, not on engine, because it is a property of the serving host rather than of how the model was trained. The same recipe is legitimately deployed under different budgets, and a template shared between deployments must not carry one of them. A config that sets engine.infer_limit or engine.infer_limit_batch_size is refused with a pointer to these deployment fields.
• No preset sets it, deliberately. A value guessed without knowing the serving hardware either binds on nothing or empties the roster on a slower box. real_time bounds latency structurally instead: one model, capacity-capped boosters, a compiled core.
• Measure before you choose one. measures.json records the strongest model's real end-to-end cost (infer_cost_s_per_row, feature engineering and model), which is the only honest basis for a number here; see Inference & fast scoring. Beware that the ceiling AutoGluon enforces is not the cost you pay: it compares a bag's per-child latency (predict_1_time / num_children), so on a preset that serves the bag itself (refit_full off) an 8-fold bag is measured at roughly an eighth of what serving it actually costs.

Per-row cost is batch-size dependent, so inference_latency_batch_size should mirror the batch the production caller actually sends; it is inert unless a budget is set.

Treelite accelerator

When deployment.deliver_treelite=True (the real_time preset sets it) ml4 attaches a compiler-free Treelite/GTIL model core that scores an eligible tree booster faster than the native predict path, transparently, with identical output.

from parleml_ml4.config.presets import real_time

model = AutoSupervised(real_time()).fit(train_df)
model.treelite is not None        # attached only when the strongest model is eligible
model.save("out/rt_bundle")       # the GTIL checkpoint rides inside the bundle

Eligibility and guarantees:

• Needs the realtime extra (pip install "parleml-ml4[realtime]"); absent it, the build is skipped and the model ships native-only.
• Eligible-only: exactly one LightGBM or XGBoost gbtree/dart booster. Bags, weighted ensembles, CatBoost, glass-box, and foundation strongest models keep the native path.
• Parity-guarded: the accelerated model core is checked against the native booster on a real sample before attaching, with an absolute bound of 1e-6 (rtol=0; scaled by the output's magnitude for regression). Any mismatch means native-only; it never raises.
• Degrades safely: a runtime GTIL failure falls back to the native predictor, so acceleration can never break serving.
• One core for every surface: the raw-frame predict / predict_proba and the encoded-frame predict_*_from_encoded (MLI, justify, ICE/LOCO, Batch Predict) score through the same GTIL model, so a bundle never serves two different cores depending on which entry point a caller used. Per-member scoring (member=) is the one deliberate exception and stays native.

The accelerator serializes inside layers.pkl, and the bundle also drops a portable standalone checkpoint (treelite_model.bin) for future compiled serving.

Airgapped export

export_python_pipeline compiles a single-model strongest model into a dependency-light ExportedPipeline that imports no AutoGluon at serve time. It replays the whole inference path (global column coercion, the fold-local chain, calibration, decision thresholds, and the target-transform inverse) in pure pandas plus the leaf library.

from parleml_ml4.delivery.export import export_python_pipeline, ExportedPipeline

exported = export_python_pipeline(model, verify_on=sample_df)
exported.save("out/scorer.xz")                 # one LZMA-compressed pickle

scorer = ExportedPipeline.load("out/scorer.xz")   # loads with no AutoGluon
proba = scorer.predict_proba(X)

It verifies parity against the AutoGluon pipeline (proba and labels for classification, point predictions for regression) before returning, raising ExportError on a mismatch.

Either calibration family exports cleanly

Neither calibration map needs AutoGluon or torch to load or to predict: temperature scaling serves as a numpy/scipy softmax and the one-vs-rest family as scikit-learn, so a multiclass run that resolves calibration_method="auto" to temperature exports airgapped as it is. See Probability calibration.

Pass a verify_on sample that contains your missing values

Missing object cells are the one place fit and serve can silently disagree, so they are worth exercising deliberately in the parity sample. _verify_matches is the backstop, and it can only check what your sample contains. A verify_on frame that carries missing values in your object columns is worth more than a larger one that does not.

Details: how the export fills missing object cells

On the fit side AutoGluon's pipeline fills every object column's missing cells with "" before the recipe chain sees them (FillNaFeatureGenerator, fillna_map={R_OBJECT: ""}). The export bypasses that pre-generator entirely (it replays the fitted ml4 transformers directly in pure pandas), so it reproduces the fill itself, normalising None / pd.NA / NaT / np.nan alike to "". For a declared categorical or text column the planner has put a CleanEncode in front, and "" is one of its NA tokens, so those cells become NaN again on both sides. For an undeclared column, or a hand-written recipe, "" is the level the chain trained on and it is preserved. Both cases match the fit side exactly.

Supported vs deferred

Strongest model	Airgapped export	Notes
Glass-box GLM / scorecard / RuleFit	supported	linear leaf (scikit-learn / imodels)
LightGBM (incl. monotonic GBM)	supported	raw lightgbm.Booster leaf
CatBoost	supported	raw catboost model + AutoGluon's categorical code map
XGBoost	deferred	its one-hot generator is an AutoGluon object; reproducing it means re-implementing that encoder
EBM	deferred	its leaf subclasses an AutoGluon model, so the artifact could not stay AutoGluon-free
GBM:arith (arithmetic features)	refused	the model's own input includes columns AutoGluon generates inside it, which the export (it replays the feature recipe only) cannot rebuild; the refusal says so and points at the bundle. See AutoGluon's Prep options
Ensembles / bagged	deferred	export supports a single model only

Deferred strongest models raise

Exporting an unsupported strongest model raises ExportError. For those, serve the full bundle (FittedPipeline.save), or steer the roster to a supported strongest model. The Treelite accelerator is an alternative only for an XGBoost strongest model. It needs a single LightGBM/XGBoost booster too, so it is unavailable for exactly the EBM / bagged / ensemble strongest models the export also refuses. If the strongest model is an ensemble that barely beat the best single model, an advantage margin is the targeted fix: it keeps the export path open without narrowing the roster. AutoGluon's own compile() is not an alternative: it optimises inference inside the predictor and still requires AutoGluon at serve time.

Choosing a deployment path

Path	AutoGluon at serve time	Any strongest model	Best for
Bundle (save / load_pipeline)	required	yes	full-fidelity serving, MLflow
Treelite accelerator	required	eligible boosters only	low-latency tree serving
Airgapped export	not required	single GLM / scorecard / RuleFit / LightGBM / CatBoost	lightweight / offline serving

See Inference & fast scoring for warmup / cost measurement.

Next: MLOps · Reproducibility & seeds