Forecasting¶

Forecaster is ml4's entry point for time-series forecasting. You give it a ForecastConfig and one long-format frame. It profiles each series, races a curated set of models under rolling-origin cross-validation, checks every winner against a seasonal-naive baseline, and returns point forecasts with prediction intervals.

In short

Pass one long frame (unique_id, ds, y) and a ForecastConfig with horizon and, ideally, freq. Irregular event data needs resample_to first.
Pick one of four backends with config.backend: stats (default), tree, foundation or autogluon. There is no cross-backend auto mode.
Each series gets its own winner, chosen on rolling-origin CV. The most recent window is held out as the honest test score: read result.topline_test().
A winner that does not beat seasonal-naive is reported on result.weak_series. It is never swapped silently.
Run the zero-shot Signal check first when you are not sure a dataset is forecastable at all.

import pandas as pd
from parleml_ml4.forecasting import Forecaster
from parleml_ml4.config.forecasting import ForecastConfig

cfg = ForecastConfig(horizon=14, freq="D")
model = Forecaster(cfg).fit(train_df)        # long frame in
future = model.predict(h=14)                 # long frame out

Optional extra

Forecasting ships as an optional dependency group.

The stats backend runs on the light pip install parleml-ml4[forecasting] (statsforecast, plus statsmodels for the profiler's seasonality measure).
The tree, foundation (Chronos) and autogluon backends build on AutoGluon-TimeSeries. They need pip install "parleml-ml4[forecasting,forecasting-autogluon]" on Python 3.11–3.13.

A run that never selects an AutoGluon-TimeSeries candidate does not import it. A backend you request without its package installed fails fast with a pip install hint.

The input frame¶

Forecaster.fit accepts a pandas DataFrame or a pre-validated SeriesFrame. The frame is long-format (Nixtla style) with three required columns:

Column	Type	Meaning
`unique_id`	str / int	Series identifier: one label per time series.
`ds`	`datetime64`	Timestamp. Must be monotonic non-decreasing per series.
`y`	numeric	The observed target. Finite (no NaN/inf) at validation time.

You can add exogenous columns (declare them in config.exog_cols) and a static-covariate frame keyed by unique_id. Rows with a blank trailing target that cover the horizon form the known-future regressor frame.

Details: how much of a future frame each series must carry

Every series needs at least config.horizon trailing blank-target rows, with the exogenous columns filled.

Rows beyond the horizon are truncated. The fit forecast, Forecaster.predict() and the deployed pipeline all head-slice the stored frame the same way.
Missing rows cannot be invented. A short series raises FutureFrameError, naming the series and the number of rows it supplied.
A series whose target is null for its whole length raises SeriesFrameValidationError; it is not dropped. All of it is "trailing", so no history is left to fit once the future frame is split off. The same frame without exog_cols gets the same rejection, so declaring a regressor never turns it into a silent deletion.

To check a frame before fitting, call parleml_ml4.forecasting.preflight.preflight_forecast(X, config). It runs the fit's own split, grid and alignment, and raises what the fit would raise, including an off-grid future timestamp (2024-02-28 on a month-end grid in a leap year). Otherwise it returns:

series_without_future: the series whose known-future regressors would be NaN over the whole horizon, because they carry no future rows at all.
calibration_shortfalls: the requested interval levels the panel's selection windows cannot calibrate, with the cv_n_windows that would.

Details: known-future vs historical-only exog on the tree backend

The tree (RecursiveTabular) backend consumes known-future exog (config.future_exog_cols, or all of exog_cols when that is None) and static covariates.

Historical-only exog are columns declared in exog_cols but excluded from future_exog_cols. The tree backend lags them forward at >= max(horizon, min_lag). They enter the model as ordinary numeric features named {col}_lag{k}.

The lag floor is what makes this safe. A lag of at least the horizon only references a value the model has already observed when the forecast is made. So these columns need no future values: supply nothing for them at predict time. The contemporaneous column itself is never a feature.

Only numeric historical-only columns can be lagged. A non-numeric one is excluded and named in the run notes, as is a declared column that is absent from the frame.
stats drops historical-only exog outright, because statsforecast cannot lag one. It records _stats_exog_dropped when it does.
AutoGluon-TimeSeries lets them ride the training frame as implicit past covariates.

The time grid¶

Every backend needs a regular, gap-free series. Before anything is fitted, the preprocessor puts the whole panel on one grid. That grid defines "one step": horizon, season_length and every lag count in those steps. Getting it right matters more than any single hyperparameter.

Three settings decide it, in this order of precedence:

Setting	Meaning
`config.resample_to`	Roll the panel up onto this grid. Defines it outright; nothing is inferred.
`config.freq`	Assert the panel's own cadence, as a pandas offset alias.
(neither)	Infer it: `pd.infer_freq` per series, majority vote across the panel.

resample_to and freq are two declarations of one fact. Setting both to different values raises.

Details: why the frequency matters even to models that ignore timestamps

Foundation models like Chronos never read a timestamp; they consume a sequence of values. The frequency is needed by everything around the model:

to turn irregular rows into a sequence at all;
to label the forecast's output dates;
to resolve season_length, which is both the MASE denominator and the SeasonalNaive baseline.

A wrong grid shifts the forecast and also moves the yardstick.

Rolling up irregular data — `resample_to`¶

Real timestamps are often events, not observations: one row per signup, per transaction, per sensor ping. There is no cadence to infer. Aggregate onto a grid you choose:

# 6,000 signups at irregular timestamps -> a weekly signup count, which IS a series.
cfg = ForecastConfig(horizon=8, resample_to="W", resample_agg="sum")

resample_agg is your call, because it encodes what the quantity is:

Aggregation	Right for	Example
`sum` (default)	flows: things that accumulate over the bucket	sales, signups, click counts
`mean`	levels: things that have a value at a point in time	price, temperature, inventory on hand
`max` / `min` / `median` / `first` / `last`	peaks, troughs, snapshots	daily high, closing price

A wrong resample_agg will not fail a metric

Summing a level (or averaging a flow) rescales the target without changing its shape. Scale-free metrics like MASE score it exactly the same. The verdict stays right and the forecast magnitudes are wrong. Only you know which kind of quantity it is.

Details: covariates, empty buckets and the future frame

Covariates are aggregated by dtype: numeric regressors average over the bucket, non-numeric ones take the first value. Static covariates are per-series constants, so they are untouched.
Empty buckets come back as ordinary gaps and go through fill_strategy (below). Resampling never invents values.
The future frame is rolled up with the history before it is checked against the horizon, because horizon counts steps of the resulting grid. Supply the future rows at your raw grain and the engine aligns them.

Gaps and `fill_strategy`¶

Once the grid is set, the preprocessor reindexes each series onto a complete date range and fills the missing timestamps. config.fill_strategy picks how:

Strategy	Fills a gap with
`None` (default)	auto: `zero` when `resample_to` is set with `resample_agg="sum"`, else `ffill`
`ffill`	the last observed value
`zero`	`0.0`
`drop_series_if_gap_pct`	`ffill`, but drops series gappier than `drop_gap_pct_threshold`

An explicit fill_strategy is always honoured. The resolved value is recorded on the persisted config.

Details: why the default follows resample_agg

The two settings together decide what an empty bucket means:

resample_to set and resample_agg="sum" → zero. Summing a flow makes an empty bucket a genuine zero: a week with no signups had zero signups.
Anything else → ffill. An empty bucket is a missing reading, and the last observation is the honest stand-in.

The wrong pairing looks plausible. sum + ffill carries the previous bucket's total forward: on the rollup example above, a week with no signups is credited with last week's 380. That inflates the level, destroys the intermittency structure the Croston models exist for, and biases every metric computed on it.

A coarse series in a finer panel is mostly fabricated, not merely gappy

The panel grid is the majority frequency. A monthly series inside a daily panel is on-grid, so nothing is discarded. But reindexing turns 6 real observations into 152 rows, about 96% of them invented by ffill. That flat plateau scores a perfect MASE, wins its race, and looks like the best series in the panel.

Fix the input, not the flag: fit the coarse series separately, pin config.freq, roll the panel up to the coarse cadence with config.resample_to, or set fill_strategy="drop_series_if_gap_pct" to drop it.

Details: how fabricated series are detected

The plateau collapses both the MAE numerator and the seasonal-naive scale, which is why it scores a perfect MASE.

config.drop_gap_pct_threshold (default 0.30) is read under every fill_strategy, not only drop_series_if_gap_pct. Any series whose imputed share exceeds it is reported as a mostly_imputed weak series and noted under result.failures["_mostly_imputed"].

result.preprocessor_summary carries max_filled_pct and p95_filled_pct alongside mean_filled_pct. The mean alone hides an outlier: one 0.96 series beside two clean ones averages to 0.32.

Details: why every fill is causal, and why there is no interpolation

Filling happens once, over the full history, before cross-validation. That is sound only because every strategy is causal: a fill that looks strictly backwards produces the same values whether you impute the whole panel and then split, or split and then impute.

Linear interpolation is two-sided. A gap spanning a CV cutoff would rebuild pre-cutoff (training) rows from post-cutoff (evaluation) values, flattering the scores of the gappy series. pandas has no causal variant (limit_direction="forward" still averages in the next observed value). So fill_strategy has no interpolation option, and a config naming interp raises. Use ffill, or drop_series_if_gap_pct if a gappy series should not be modelled at all.

A two-series daily panel:

import numpy as np
import pandas as pd

dates = pd.date_range("2023-01-01", periods=365, freq="D")
train_df = pd.concat([
    pd.DataFrame({"unique_id": "store_A", "ds": dates,
                  "y": 100 + np.arange(365) * 0.2 + np.random.default_rng(0).normal(0, 5, 365)}),
    pd.DataFrame({"unique_id": "store_B", "ds": dates,
                  "y": 50 + 10 * np.sin(np.arange(365) / 7)}),
], ignore_index=True)

model = Forecaster(ForecastConfig(horizon=28, freq="D")).fit(train_df)

The four backends¶

config.backend selects the model family explicitly: stats (default), tree, foundation or autogluon. There is no cross-backend "auto". You pick the backend, and config.mode then chooses within it: race its candidates, or pin one model. A series that does not beat seasonal-naive is reported as a warning; it is never swapped to a heavier backend.

Backend	Library	Pick it for
`stats`	statsforecast	Fast, robust classical models (19): `auto_arima`, `auto_ets`, `auto_theta`, `mstl`, `seasonal_naive`, the Croston intermittent-demand family and more. Strong on many or short series; the safe default.
`tree`	LightGBM / XGBoost / CatBoost, through AutoGluon-TimeSeries `RecursiveTabular` (which wraps mlforecast)	Explainability and native exogenous drivers. See Interpreting a tree forecast.
`foundation`	Chronos, through AutoGluon-TimeSeries	Zero-shot, no training, fast. Strong out of the box for cold-start or many series. Point-only: bands come from the conformal layer (Prediction intervals).
`autogluon`	AutoGluon-TimeSeries	Highest accuracy, through a weighted ensemble over a model zoo (Chronos-2, Toto-2, DeepAR, TFT, tabular, classical). Slowest; covariate-aware.

Details: what the tree and autogluon backends add

tree. It explains through the same MLI ProfilingResult stack as classification: gain and SHAP in target units, representative trees, archetypes, PDP/ALE and rules, on the Explain tab. The AutoDoc model card carries them too. It adds forecast-specific SHAP views: a per-(series, date) waterfall, and importance by horizon and by series. ml4 owns the feature plan and the orchestration (Optuna, CV, conformal). RecursiveTabular executes the numba recursion and yields a bare booster that mli explains natively. It uses the pinned AutoGluon-TimeSeries 1.6.3 (Python 3.11–3.13 within ml4).

autogluon. Pick an AutoGluon-TimeSeries preset with autogluon_preset: medium_quality, high_quality, best_quality, experimental_quality, the Chronos presets, or AutoGluon's short aliases. fast_training and very_light are refused, with the preset to use instead. Restrict the zoo and the time budget with autogluon_models and autogluon_time_limit_s. Left unset, the preset follows budget: balanced → medium_quality, thorough → high_quality.

Details: adding a backend

Backends are registered in parleml_ml4/forecasting/backends/_registry.py. Every backend and both modes run through one fit path and one cross-validation runner (parleml_ml4/forecasting/_cv.py). Both read the backend's facts, not its name.

What depends only on the backend is stated on the class:

Fact	Means
`prepare_frame(sframe)`	the Nixtla frame this backend is fitted and cross-validated on (only `tree` merges static covariates)
`emits_native_fit_levels`	whether `fit` honours `level` and emits analytic bands, so CV asks for them (only `stats`)
`cv_is_atomic`	whether one `cross_validation` call is all-or-nothing, so a failed pass is retried once at one window (only `stats`)
`min_train_size(config, season_length)`	the rows a series needs before a CV cutoff for this backend to forecast it, counted into every fold grid's eligibility and its one-window fallback (1, except `tree`: its longest lag plus one)
`apply_time_budget(remaining)`	takes the wall clock left after CV before the final fit; `True` when none is left (only `tree` spends it)

What depends on the run is a field of the registry entry:

Field	Means
`build`	construct the backend from the config (optional dependencies fail fast here)
`route`	the auto-mode candidates for a profile: the backend's `route_*` function in `parleml_ml4.forecasting.router`
`pin`	the single-mode spec for `model_name` (on `stats`: the resolved season, AutoARIMA's approximation rule and the AutoETS / AutoCES clamp, exactly as a routed spec gets them)
`route_notes`	notes about a routed roster itself (`stats`: the season clamp, a trimmed multi-seasonal branch)
`collapse_spec`	what to ship instead when an auto race produces no CV evidence (only `stats`: SeasonalNaive)

A new backend is therefore: write the class with its facts, add one registry entry, and extend the ForecastConfig.backend Literal. The orchestrator does not change.

Choosing a foundation checkpoint¶

config.foundation_model_name picks one of two Chronos families:

	Chronos-2	Chronos-Bolt
Names	`chronos2` (default), `chronos2_small`	`chronos_bolt_{tiny,mini,small,base}`
Exogenous regressors	Yes: known-future and past-only	No, univariate
Native prediction length	1024 steps per pass	64
Context length	8192	2048
`foundation_dtype`	not applicable: it selects its own precision	honoured
Pick it for	the default: accuracy, covariates, long horizons	the lightest CPU option

Both are Apache-2.0 and ship inside AutoGluon-TimeSeries, so neither needs an extra dependency. foundation_device, foundation_batch_size and foundation_context_length apply to both. foundation_dtype applies to Chronos-Bolt only and is not sent to Chronos-2.

The weights must be on the server¶

AutoGluon loads each checkpoint's weights from the Hugging Face cache and downloads them when they are missing. A foundation backend checks the cache when it is built, before any window fits. When the weights cannot load it raises FoundationWeightsMissing, naming the checkpoint, the ones that are cached, and the remedy.

Details: checking and warming the weights

parleml_ml4.forecasting.foundation_weights answers from the cache alone (no network, no torch). A checkpoint is cached, download (not cached; the first run fetches it) or missing (not cached, and HF_HUB_OFFLINE forbids the download).

from parleml_ml4.forecasting.foundation_weights import foundation_weights, require_foundation_weights

foundation_weights("chronos2").status        # "cached" | "download" | "missing"
require_foundation_weights("chronos2")       # raises FoundationWeightsMissing when "missing"

To warm an offline install, run this where Hugging Face is reachable, with the same HF_HOME:

python ml4/scripts/fetch_foundation_models.py --kind forecast --models chronos2,chronos2_small

The Signal Check probes with chronos2_small. To compare a foundation run with its number, set foundation_model_name="chronos2_small", or name the difference in checkpoint.

AutoGluon treats every model as optional and skips one that fails. When it trains no model at all, the fit raises ModelsNotTrained with AutoGluon's own reason (the error its trainer logged). You get the error at the fit, not at the first prediction.

On Chronos-2, foundation_batch_size changes the forecast

Chronos-2 uses cross-learning by default: a series borrows context from the others in its batch. Changing foundation_batch_size, or adding an unrelated series to the panel, changes results. Set config.cross_learning=False when you need series-independent forecasts.

Details: cross-learning and reproducibility

Cross-learning is usually an accuracy win on a panel, which is why the fit path leaves it on. But a series' forecast then depends on which other series share its batch. Upstream recommends a batch size near 100 (the value in the Chronos-2 technical report), not a throughput-maximal one.

config.random_state does not make this reproducible: batching follows item order, and AutoGluon's seed does not control batch composition. The Signal Check turns cross-learning off, because it scores a capped subset of the panel.

Race vs single (within the chosen backend)¶

config.mode chooses the orchestration inside the backend you picked:

auto — race the candidates (default)single — pin one model

The SeriesProfiler characterises each series (seasonality, intermittency, trend), and the backend's router curates its candidate list. Rolling-origin cross-validation then picks the per-series winner on the primary metric.

cfg = ForecastConfig(
    backend="stats", mode="auto",     # race the stats candidates
    horizon=14, cv_n_windows=3,
)

Fit one named model in that backend, with no race.

cfg = ForecastConfig(
    backend="stats", mode="single", model_name="auto_ets",
    horizon=14,
)

Single mode requires model_name, and the config checks that the name belongs to the chosen backend.

It is still cross-validated: the weak-series check and the conformal intervals are computed from that. On stats the pinned model gets the season treatment a routed one gets: the resolved season_length, the CSS approximation for AutoARIMA above season 12, and the AutoETS / AutoCES clamp above 24 (reported as _season_clamped_candidates). A value you set in model_kwargs always wins.

What the profiler measures, and what the router does with it¶

The routing decisions that remove candidates are panel-level, so they use panel-level statistics, not the most extreme series:

Property	Measured as	What it gates
Seasonal strength	the bias-corrected decomposition variance ratio at each candidate period, averaged over every sampled series; `0.0` unless a panel-wide seasonality test is significant	the resolved `season_length`: the MASE denominator, the SeasonalNaive baseline, and the seasonal branches
History length	the panel median	whether the statistical families (AutoETS / AutoARIMA / AutoTheta / AutoCES) race at all, and `WindowAverage`'s window
Intermittency	share of zeros per series, majority vote, net of any zeros `fill_strategy="zero"` invented	whether the roster switches to the Croston / ADIDA / IMAPA family

Everything the router keeps stays per-series. Cross-validation picks the winner per series, and statsforecast enforces each model's own length requirements per series. Clamps and declined branches are reported in result.failures (see the run notes).

Details: how seasonality is tested

Separating trend from season is the hard part. The measure is the textbook one: the share of de-trended variance the seasonal component explains, over a classical decomposition (statsmodels.seasonal_decompose). An autocorrelation would read any trend, straight or curved, as seasonality at every lag.
Noise must not buy a season. The share is bias-corrected (like an adjusted R²). A period counts only when a seasonal-dummy F-test, combined across the sampled series, rejects "no seasonality" (α = 0.01; 0.10 for the frequency's own textbook cycle). Multiples and aliases of an accepted shorter cycle are not credited again. Of two periods less than 2× apart, only the stronger is kept.
A period needs three full cycles to be tested. A sampled series too short for it scores zero rather than abstaining, so a long period is not judged only on the series long enough to support it. A period no sampled series can test is omitted. That is how "untested" (fall back to the frequency's textbook cycle) differs from "tested and absent" (season_length = 1).

tree: tree_target_transform decides whether it can follow a trend

Leave config.tree_target_transform at auto unless you need SHAP in target units. Without a target transform, a tree forecast flattens at the last observed level on a trending series.

Details: tree_target_transform values and why the default matters

A gradient-boosted tree predicts a leaf mean, so without a target transform its forecast is bounded by the range it trained on. On a trending series the recursive forecast flattens at the last observed level and stays there for the whole horizon. And because the tree backend fits one global model pooled across every unique_id, an unscaled target lets the largest-scale series dominate the squared-error objective while the small ones are fitted as noise.

The default auto defers to AutoGluon's own choice, a standard scaler plus seasonal differencing, which prevents both.

Value	Effect
`auto` (default)	AutoGluon's defaults: standard scaler + seasonal differencing
`none`	raw `y`. SHAP and the reconstructed design matrix are in target units, and `lag1` equals the previous observation exactly: the legible choice, at the cost of trend extrapolation
`standard` / `mean_abs`	scaler only
`difference`	seasonal differencing only, at ml4's own resolved `season_length`

Under anything but none the booster is fitted on a transformed target, so the importances and SHAP values are in transformed units. The run records that on result.failures (_tree_target_transform).

Interpreting a tree forecast¶

The tree backend fits a native gradient-boosted regressor over a lag-feature matrix. So it flows through the same MLI ProfilingResult stack as classification (importances, SHAP, representative trees, archetypes, PDP/ALE, rules) on the Explain tab, and the AutoDoc model card carries the same sections. Three forecast-specific views live on the detail panel (tree backend only):

Shapley: a per-(series, date) SHAP waterfall. Pick a series and a prediction date to see which lag, calendar or exogenous features moved that one forecast. Future dates seed the lag context from the model's own recursive forecast.
Feature importance → By horizon: a heatmap of the |SHAP| share at each forecast step h=1…H. The model forecasts recursively, so reliance shifts across the horizon: recent lags dominate the near term and give way to calendar and longer-memory features further out.
Feature importance → By series: a heatmap of the |SHAP| share per unique_id (each row a scale-free feature distribution), so you can compare which drivers matter for which series.

Statistical and foundation forecasts expose no per-feature model, so these views are hidden for them. The Model selection heatmap summarises their raced candidates instead.

Details: units and weighting of the SHAP views

Under the default tree_target_transform="auto" the booster works in a transformed space (per-series scaled seasonal differences). SHAP values and lag features are in that space, and the chart names it on its axis.
By horizon normalises each series to a unit total before averaging. Every series gets equal weight, so a high-volume series cannot dominate when SHAP is in raw target units under tree_target_transform="none".

Per-series cross-validation¶

Model selection runs rolling-origin cross-validation with cv_n_windows windows (default 3; the thorough budget raises it to 4). Each series is scored independently and gets its own winner, on the earlier windows only. The most recent window is held out of selection as an honest test (see Metrics). With cv_n_windows = 1 there is no window to hold out, so selection uses the single window and no held-out test is reported.

Backtest geometry — one grid for every CV site¶

All fold geometry comes from one shared grid (parleml_ml4.forecasting.splitting.CVGrid), used by every backend and by the tree backend's Optuna inner folds. The resolved geometry ships on the result as result.cv_grid. In short:

The last window ends each series; earlier windows step back by cv_step_size (default: the horizon).
A series races only if it is long enough for every window. Shorter series are still forecast, with a default winner, and reported.
cv_gap drops rows before each cutoff from training (an embargo). The stats backend cannot honour it.

Details: the full grid contract

Per-series anchoring. The last window's validation block is the final horizon observations of each series. Earlier origins step back by cv_step_size. Series of different lengths each get their own cutoffs.
All-or-nothing eligibility. A series participates in the race if and only if it is long enough for every window (horizon + step·(n_windows−1) + gap + min_train_size rows). min_train_size is the chosen backend's own floor on the rows before a cutoff: 1 on stats, foundation and autogluon, and the longest lag of the config-derived plan plus one on tree (a window whose training slice has no warmed-up lag has no row to fit on). So at horizon=12 on monthly data with a yearly cycle, a tree series needs 37 rows for one window, not 13. Shorter series are excluded from model selection only. They are recorded under failures['_cv_short_series'] and still forecast with a default winner label. No backend scores a series over a different number of windows than another.
One-window fallback. When no series is long enough for the configured windows, every backend cross-validates on a single window instead, by the same measure, and the run says so under _cv_windows_incomplete. There is then no held-out test window.
One default label. A series that carries a winner it did not win (too short for the grid, or every candidate non-finite) is labelled SeasonalNaive when SeasonalNaive raced, else the first candidate, on every backend. It is reported as a not_validated weak series.
Embargo (cv_gap). The gap rows before each cutoff are excluded from that window's training slice. The tree backend recurses across the gap, and AutoGluon-TimeSeries and foundation forecast gap + h steps across it. statsforecast cannot honour the embargo (its internal CV trains through to each cutoff), so the run records failures['_cv_gap_unsupported_stats'] once.

tree: rolling_method decides whether the intervals mean anything

The default rolling_method="tta" fits only the earliest CV window and reuses its weights for every later one. The conformal bands are calibrated on those residuals, so under tta the intervals carry no finite-sample coverage guarantee. If you act on the interval widths rather than the point forecast, set rolling_method="refit". It costs cv_n_windows fits.

Details: rolling_method, calibration basis and tta vs refit

Window 0 is the shortest history in the run, so under tta most windows are scored by an under-trained model. Split conformal's coverage guarantee assumes the residuals come from the predictor being banded. So under tta the published intervals carry no finite-sample validity claim, and the run records a note saying so.

The same fact is typed on every conformal level of the result. result.intervals[i].calibration_basis is "proxy", and calibration_proxies names why: "tta" (earliest-window weights), "untuned" (default hyper-parameters while the final fit ran a search), or both. Every summary surface reads it as approximate coverage. The backtest coverage is its empirical check. Only rolling_method="refit" with budget="fast" (no search) gives "deployed_model". ml4 does not calibrate the shipped bands on a CV pass that runs the tuned configuration.

Why tta can score better than refit. Both paths build each window's features and target transform from that window's own history: mlforecast's predict(new_df=…) refits the per-series scaling and differencing on it. So they differ only in the weights. On the raw target (tree_target_transform="none") a trend carries later windows past the earliest model's training range, and tta degrades with distance. Under the default "auto" the earliest model forecasts a stationary, scale-free residual, so tta loses little and the two land within noise of each other. On a single series either can come out ahead. A tta run that beats refit is not evidence that the earliest window generalises better.

Details: tree CV scores an untuned model, and the search is seeded

CV scores an untuned model. Every tree CV window is scored with the default hyper-parameters and the config-derived feature plan. The model that ships is the one the Optuna search picked; a successful search replaces both the model kwargs and the whole feature plan. Tuning per window would multiply the tuning budget by the window count, and tuning once up front on the full frame would leak the validation windows into the search. So a tree run's CV numbers (the model-selection heatmap, the AutoDoc card, the weak-series check) describe the untuned shape of the winning candidate, and the deployed model is normally at least as good. The run logs this once whenever tuning is on: any budget other than fast, which runs 0 Optuna trials.

The search is seeded. The Optuna sampler is seeded from random_state (default 0). Re-fitting the same frame under the same config deploys the same lags and the same hyper-parameters; change random_state to explore a different trajectory. Because a successful search replaces the whole feature plan, an unseeded sampler would ship two different models from two identical fits.

Honest warnings (weak series)¶

There is no silent SeasonalNaive swap and no cross-backend escalation. You picked the backend, so ml4 ships its per-series winner and tells you where that winner is weak.

A series is flagged on result.weak_series (a tuple[WeakSeries]) when its deployed winner cannot be trusted. Strongest reason first:

mostly_imputed: more than config.drop_gap_pct_threshold of the series' rows were fabricated by fill_strategy when it was put on the panel grid (see Gaps and fill_strategy). No score on such a series means anything, so this reason outranks the others.
degenerate_scale: the MASE is not finite, because the seasonal-naive error it is divided by is (near) zero on a flat training window. The metric is undefined, not bad. Check whether the series is constant, or too short to vary.
not_validated: the series' model was assigned, not raced. The series was too short for the grid, no candidate produced a comparable score on it, or the run's CV produced no evidence at all. Its winner is the run's default label (SeasonalNaive whenever SeasonalNaive raced, otherwise the first candidate). It is never counted as a SeasonalNaive win.
seasonal_naive_won: SeasonalNaive itself won the race (nothing beat the naive baseline).
below_seasonal_naive: the winner's CV MASE > 1.0 (worse than the in-sample seasonal-naive it is scaled against).

The check works the same way on every backend. The Summary payload, the forecast detail panel and the AutoDoc card all show the count and the flagged series, grouped by reason.

Details: which score the weak-series check reads

The check reads both score estimates (the selection score and the held-out test score), fires on the worse of the two, and reports that number as mase.

The selection score alone is not evidence. It is measured on the windows the winner was chosen on, so with six or seven candidates racing it is optimistically biased: a winner whose true MASE is above 1 can post a selection MASE below it. A single held-out window is unbiased but noisy. Taking the max is the right trade for a check that can only add a warning, never remove a real one.

Run-level notes land on result.failures under _-prefixed keys. The fit succeeded, so these are breadcrumbs, not errors.

Details: every run note a fit can record

Notes the orchestrator raises go through one recorder (Forecaster._record_note, first writer wins). Notes a backend raises go through its run_notes_ dict and are merged at one seam. The wording lives in parleml_ml4/forecasting/_honesty.py or beside the backend that raises it. The list below is the complete set.

The data was not what the model needed

Key	Means
`_mostly_imputed`	at least one series is mostly fabricated by `fill_strategy` (above)
`_cv_short_series`	series too short for the CV geometry (the windows, plus the rows the backend needs before a cutoff: the tree backend's longest lag) were excluded from selection
`_cv_skipped_series`	series that produced no CV rows at all, so they carry a default label (and are `not_validated` weak series)
`_seasonal_naive_wins`	SeasonalNaive won on some series: nothing in the roster beat the yardstick
`_selection_defaulted_series`	a series was ranked, but every candidate scored non-finite, so its model was assigned rather than won
`_selection_unscored_series`	a series was cross-validated and still produced no score row to rank
`_profile_basis`	routing and `season_length` were measured on the full history, evaluation windows included

The backtest is smaller or weaker than requested

Key	Means
`_cv_windows_incomplete`	fewer windows scored than were asked for: windows failed, or the panel was too short and was cross-validated on one; every derived number shrinks with them
`_single_selection_window`	`cv_n_windows=2` leaves one window for selection once the honest test is held out
`_cv_gap_unsupported_stats`	`cv_gap` was requested but statsforecast trains through to each cutoff
`_validation_window_missing` / `_test_window_missing`	series absent from the validation or held-out test split, so that metric averages over fewer series
`_cv_scores_not_deployed_model`	the tree backend's CV scored a model that is not the deployed one (recorded only when a window scored)
`_<backend>_ungated`	cross-validation produced no evidence (no series has the rows one window needs on this backend, and the note says how many; CV failed; or no column for any candidate), so every series is a `not_validated` weak series. Recorded on every backend, in both modes. A `stats` auto race also ships SeasonalNaive and records the failed stage `stats_race_collapsed`
`_final_fit_untuned`	the wall-clock budget was spent by CV, so the deployed model uses default hyper-parameters

The roster shipped something other than what was diagnosed

Key	Means
`_season_clamped_candidates`	AutoETS / AutoCES raced at a shorter season than the run resolved
`_multi_seasonal_unconvincing`	more than one candidate period, none strong enough to hand a decomposition model
`_multi_seasonal_branch_trimmed`	`budget='fast'` dropped MSTL / AutoTBATS on a multi-seasonal panel
`_stats_model_fallbacks`	a per-series fit failed and shipped a Naive forecast under the requested alias
`_stats_exog_dropped`	declared regressors statsforecast cannot consume
`_foundation_covariates`	columns this Chronos checkpoint ignores
`_optuna_unavailable`	no hyper-parameter or feature-plan search ran; hand-picked defaults were used
`_tree_target_transform`	the tree backend fitted a transformed target, so its SHAP values are in transformed units
`_tree_short_series_dropped`	AutoGluon dropped items too short for the fitted target transform
`_tree_subsampling_disabled`	the panel is above AutoGluon's own sampling thresholds, so ml4 disabled them for reproducibility
`_historical_exog_lagged`	historical-only regressors were lagged forward, and any that could not be (non-numeric) are named
`_future_exog_missing`	a known-future regressor had no values at predict time and was filled with NaN
`_stats_cv_naive_cells`	a per-series statsforecast fit failed during CV, so those cells score `Naive` under the requested alias
`_foundation_device_downgraded`	the requested `foundation_device` is unavailable, so inference ran elsewhere
`_foundation_context_clamped`	`foundation_context_length` exceeds the checkpoint's maximum and was clamped

Intervals and overlays

Key	Means
`_intervals_unavailable`	no band could be produced for a level
`_intervals_uncalibrated`	a band was produced but does not carry its stated coverage (see Prediction intervals)
`_intervals_unscaled`	these series carry no band: their seasonal-naive error, the scale every conformal band is sized by, is zero or undefined (a constant history, or one no longer than a season)
`_horizon_exceeds_native`	the horizon is past the foundation checkpoint's one-pass limit, so the far end of the forecast comes from autoregressive rollover and decays with distance (see Signal check)
`_reconciliation_failed`	`hierarchy_cols` were configured but the reconciled view could not be computed; the per-series forecast is unaffected
`_reconciliation_partial`	some series were excluded from the hierarchy, so every total above them is an under-count, not a coherent roll-up
`_intervals_undercover`	the published bands were measured on the held-out window and covered materially less than their stated level
`_interval_coverage_not_measured`	`cv_gap > 0` shifts the calibration residuals' step labels, so coverage was not measured (the bands are unaffected)
`_intervals_native_quantiles`	the AutoGluon-TimeSeries bands come from the model's own quantile heads, so the conformal calibration checks do not apply to them
`_stats_intervals_conformal`	the `stats` roster includes a model with no analytic interval, so the whole roster ran point-only and the conformal layer bands the winner

A weak series is a signal, not a bug

A flagged series' forecast is not trustworthy as a modelled prediction; the series may not be worth forecasting at this horizon. Try a different backend (tree for exogenous drivers, autogluon for the ensemble), revisit the horizon or the aggregation grain, or accept the naive baseline. result.weak_series records which series and why.

Prediction intervals¶

Point-only winners get conformal prediction intervals, calibrated from the CV residuals. Models that emit their own bands keep them. The levels come from config.confidence_levels (default (80, 95)), and you can override them at predict time:

future = model.predict(h=14, level=(80, 95))
# columns: unique_id, ds, <model>, <model>-lo-80, <model>-hi-80, <model>-lo-95, ...

Where each backend's bands come from:

Backend	Where the bands come from
`stats`	analytic, from the model itself, when every model in the roster has an analytic interval (AutoARIMA, AutoETS, AutoTheta, MSTL, Naive, SeasonalNaive, HistoricAverage, RWD). When the roster includes any of the nine with no analytic form (the Croston family, ADIDA, IMAPA, TSB, WindowAverage, SeasonalWindowAverage, AutoMFLES), the whole roster runs point-only and ml4's conformal layer bands the winner (`_stats_intervals_conformal`)
`tree`	conformal (the booster forecasts a point)
`foundation`	conformal, on both Chronos families
`autogluon`	native quantile heads; a pretrained member can narrow the 95% band (see below)

Point metrics cannot see an interval defect, so every run also reports, per series, on the held-out window the bands were not calibrated on:

interval_coverage_{L}: the fraction of held-out points inside the level-L band. Read it against L / 100, not against 1.0.
interval_width_{L}: the mean band width in target units. Coverage without width is not interpretable, because any band can buy coverage by widening.

A material shortfall is recorded as failures['_intervals_undercover']. Over-coverage is not flagged: split conformal is deliberately conservative at small calibration sizes.

Conformal bands come from parleml_ml4.forecasting.conformal. Their residuals are temporal, not exchangeable, so treat a nominal level as a target, not a guarantee. A forecast band is an interval for y; unlike the regression bands, it is not required to contain the point forecast.

Non-negative targets¶

Demand, counts and sales cannot go below zero, but a forecast of them can: a band around a small point on an intermittent series easily reaches below zero, and a trend model can extrapolate through it. config.non_negative sets a zero floor:

Value	Behaviour
`"auto"` (default)	Floor at zero when every training target value is >= 0.
`"on"`	Always floor at zero.
`"off"`	Never floor; publish the raw values.

The floor applies to every point and every -lo-/-hi- bound: in result.forecast, in model.predict() and in the deployed pipeline's predict(). It is also applied to the cross-validation predictions before they are scored, so model selection, the validation and test metrics and the measured interval coverage describe the values that are published. Raising a lower bound to zero cannot exclude a non-negative actual, so coverage stays the same or rises.

result.value_floor records the floor the run applied (0.0, or None when it applied none). A run that changed any value says so in its notes: _forecast_floored for the published forecast and _cv_floored for the scored backtest.

Details: why foundation is point-only on purpose

Chronos-Bolt cannot serve a level natively. It has nine trained quantile outputs (0.1 … 0.9) and clamps anything outside them onto the nearest one. The default (80, 95) asks for 0.025 / 0.975, both of which land on 0.1 / 0.9. A native "95%" band would be numerically identical to the 80% band while claiming wider coverage. So the foundation backend declares no native levels (FoundationBackend._native_interval_levels), returns a point forecast, and takes every band from the conformal layer, calibrated on the run's own holdout residuals. The AutoGluon-TimeSeries clamp warning ("Quantiles to be predicted … are not within the range of quantiles that Chronos-Bolt was trained on") therefore does not appear on a foundation fit.

Chronos-2 does pass arbitrary quantiles through, so that reason does not apply to it. Conformal is still the default there, deliberately: it is calibrated on the run's own residuals and honest at every requested level. Whether Chronos-2 honours an arbitrary quantile or interpolates between trained ones is an open empirical question.

Details: the autogluon ensemble carries a fixed-head member

AutoGluon-TimeSeries owns the zoo and the ensembling. Going point-only (the foundation approach) would discard the genuine per-level heads its trained members fit. What to know:

budget="balanced" (the default) → medium_quality → the light zoo, and budget="thorough" → high_quality → the default zoo. Both include a Toto-2 member, whose quantile head stops at 0.1 / 0.9: AutoGluon clips a requested 0.025 / 0.975 onto those knots, silently. Both also include Chronos-2, whose trained grid spans 0.01 … 0.99 and interpolates within it, so it serves the default (80, 95) levels as asked.
A pinned Chronos-Bolt member (autogluon_models=("Chronos", …) or a bolt_* preset) clamps the same way Toto-2 does, with AutoGluon's clamp warning.
budget="fast" uses no AutoGluon preset. ml4 trains a fixed cheap zoo (Naive, SeasonalNaive, ETS, Theta, RecursiveTabular, DirectTabular), with no pretrained member, so no clamp.

The effect is a 95% band biased narrow in proportion to the weight the ensemble gives a clamped member; an 80% band is unaffected. To avoid it, pin the zoo with autogluon_models to members that fit per-level heads, or use backend="tree" or backend="foundation", which are conformal end to end.

Details: what a level needs, and why the horizon does not supply it

A conformal band is an order statistic of the calibration scores, and rank selection is clipped into [1, n]. A level therefore needs a minimum calibration size before its rank stops pinning to the observed extreme: (100 + L) / (100 − L) scores, so 9 for an 80% band and 39 for a 95% one (parleml_ml4.delivery._conformal.min_conformal_residuals).

The unit that counts is one series in one calibration window, not the residual row. The h residuals inside one window come from a single recursive path off a single origin. They share that origin's state and, under the default rolling_method="tta", the same fitted booster. So a longer horizon buys rows but no exchangeability.

A series holds only cv_n_windows − 1 calibration windows (the most recent window is the honest test and never calibrates anything), so the layer pools across series:

Each residual is divided by its series' seasonal-naive error at that window's origin: the mean absolute error of repeating the value one season back, over the history up to the cutoff. That is exactly the scale MASE divides by.
The scaled scores are pooled across series.
The quantile is multiplied back by the series' own seasonal-naive error at the forecast origin.

The scale is a statistic of the history, not of the residuals. So a series of any magnitude is put on the common scale even with a single calibration window (cv_n_windows=2), and no residual shrinks its own scale. Coverage holds on average over the panel and, to the extent a series' errors are proportional to its seasonal-naive error, series by series. A series whose history has no seasonal-naive error (constant, or no longer than a season) ships unbanded and is named under failures['_intervals_unscaled'].

At the default cv_n_windows=3 that is two scores per series. An 80% band is then calibrated from about 5 series and a 95% band from about 20. A single series needs cv_n_windows = 10 and 40.

Each forecast step uses its own pooled set when that set can place the most demanding requested level; the band is then calibrated and free to widen with the step. Otherwise it uses the all-steps set, a steadier quantile of constant width. The choice is made once for all requested levels, so the bands stay nested. Levels the independent scores cannot place are reported: as failures['_intervals_uncalibrated'] on the result, inline under the forecast chart, and as a log line from a deployed model. Raise cv_n_windows or fit more series together; a longer horizon adds rows from the same windows and changes nothing.

tree and foundation are point-only, so both go through this layer. The stats backend's analytic intervals are unaffected, as are the autogluon ensemble's native quantiles. The uncalibrated report is defined over conformal bands, so it never fires on an autogluon run, whose bands have no residual-count check at all.

Details: calibration uses the selection windows only

The most recent window per series is the honest test estimate, so it is not calibration data. A run cannot both hold a window out and calibrate on it.

A single-series run at horizon=1 with cv_n_windows=3 has two residuals. That is below the floor (three) at which a quantile means anything, so it ships unbanded with failures['_intervals_unavailable'] rather than inventing a band from two points.

Holding the window out is also what makes the coverage measurement possible. Scoring a conformal band on its own calibration residuals is circular and reports near-nominal coverage by construction.

Details: the bands ride the deploy bundle

The residual source is persisted as conformal_cv.parquet independently of config.save_cv_table, so a saved model serves the same intervals the fit reported. save_cv_table=False drops only the full raw CV predictions (cv_table.parquet).
An exog run's fit-time future-regressor frame rides along (x_future.parquet), so FittedForecastPipeline.load(...).predict() reproduces result_.forecast instead of re-forecasting with NaN regressors.
The bundle carries each series' last observed timestamp, so the serve path runs the same future-frame validation as the fit without shipping the training frame.
It carries the conformal layer's seasonal-naive scales, so the served bands are sized exactly as the fit's were.
A bundle stamped with any schema version other than the running ml4's (10.0) is refused on load, with a message to re-fit.

Measuring a band's coverage on every backtest window¶

The fit measures each level's coverage on the held-out test window (result.intervals[i].test_coverage): one window, horizon points per series. For more evidence, parleml_ml4.forecasting.backtest_coverage measures it on every window of the run's backtest, without refitting.

Details: running backtest_coverage

Each validation window is banded from the residuals of the other validation windows (never its own, which would be circular). The test window is banded from all of them (the shipped band). Each goes through the same add_conformal_intervals:

from parleml_ml4.forecasting.backtest_coverage import backtest_coverage

measured = backtest_coverage(
    pipe.result.cv_table,
    levels=tuple(pipe.config.confidence_levels),
    scales=pipe.conformal_scales,
    winning_model_per_series=pipe.result.winning_model_per_series,
    gap=pipe.result.cv_grid.gap,
)
for w in measured.windows:          # window 1 is the oldest; the test window is last
    print(w.window, w.held_out, [(c.level, c.covered, c.scored) for c in w.levels])
print([(c.level, c.coverage, c.scored) for c in measured.pooled])

A validation window's band is drawn from one window fewer than the shipped band. Where the shipped band's per-step set barely reaches a level, the held-out one may not: WindowCoverage.uncalibrated_levels says so. Nothing is measured, and measured.reason says why, for winners with analytic intervals (the coverage plot scores those in the backtest itself), fewer than two validation windows, or a cv_gap embargo.

Not the regression interval layer

Prediction intervals documents the regression layer (CQR, IntervalsConfig, predict_intervals, lo_80-style columns). Forecaster does not use it: it has no predict_intervals method, and its bands are the ones described above.

Hierarchical reconciliation¶

Set ordered (coarse → fine) grouping columns in config.hierarchy_cols. The bottom-level forecast is then rolled up into coherent per-level forecasts with hierarchicalforecast's BottomUp reconciliation. The result lands on ForecastResult.reconciled_forecast in a single winner column. Flat-series runs are unaffected.

If reconciliation was configured but could not be computed, the run keeps its per-series forecast and records failures['_reconciliation_failed'] with the reason. An empty Hierarchy tab then says why.

Details: what is reconciled, and what the aggregate rows do not carry

The winner is chosen per series, so the thing rolled up is the deployed forecast: one column assembled from each series' own winning model. Reconciling each model alias separately would total a panel nobody ships, because a parent's children may each be forecast by a different model.

The aggregate levels carry no uncertainty. Reconciliation runs after interval construction and works on point columns only, so there are no bands above the bottom level. Summing the children's bands would be wrong anyway; that needs sample paths, which point reconciliation does not have.
BottomUp over these aggregates is an algebraic identity. aggregate builds each parent as the sum of its children, and BottomUp recomputes exactly those sums, so the pass changes no number. It is scaffolding for a reconciler that does redistribute (MinTrace), not a coherence check that could fail.

When a declared dependency is missing¶

holidays, hierarchicalforecast, optuna, statsforecast and the rest are declared dependencies of ml4's forecasting extra. Their absence is a broken install, not a configuration choice. Two rules follow:

A missing package that would change the model raises. Holiday features without holidays, or hierarchy_cols without hierarchicalforecast, fail the fit with an actionable message. The fit never trains on a different feature set from the one the config declared.
A missing package that only weakens the search is recorded. Without optuna the tree backend falls back to hand-picked defaults. That is a defensible model, not a different one, so the run continues and says so under failures['_optuna_unavailable'].

Metrics¶

The available metrics live in parleml_ml4.forecasting.metrics: mase, smape, mae, wape, rmsse. config.metric_primary (default mase) drives per-series selection. The CV score frame always also carries mase and smape.

Metric	Unit	Read it as
`mase`	ratio	`1.0` = the seasonal-naive baseline
`rmsse`	ratio	MASE's squared-error companion (the M5 metric), on the same scale; punishes large misses harder
`wape`	ratio	absolute error over the total absolute actual (`utilsforecast`'s `nd`)
`mae`	target units	comparable only within one series
`smape`	percentage in `[0, 200]`	the standard Makridakis convention

All of them delegate to utilsforecast. The underlying utilsforecast.losses.smape returns half the percentage as a fraction, so ml4 rescales it once at the metric boundary. Read raw, 0.150 looks like 15% and is really 30%.

These are all point metrics. Interval quality is reported separately as interval_coverage_{L} and interval_width_{L} (see Prediction intervals).

Validation (selection) vs the honest held-out test¶

Two numbers, and the difference matters:

Validation is the rolling-origin CV selection score. It is optimistically biased (winner's curse: the winner is the model that scored best on exactly this metric), so read it as a ranking, not a performance guarantee. Two views:
- result.cv_scores / topline_metric(): the per-candidate selection scores the winner was chosen from.
- topline_validation() / validation_scores: the deployed winner scored on the same selection windows. This is the number shown on the card next to the test, and it is directly comparable to it.
Test is result.topline_test() / result.test_scores: the honest out-of-sample estimate. The most recent CV window per series is held out of selection, and each series' winner is scored on that untouched window. This is the number to trust for "how good is this forecast?".

The held-out test needs cv_n_windows ≥ 2. With cv_n_windows = 1, topline_test() and topline_validation() are empty (as are test_scores and validation_scores), leaving only the per-candidate topline_metric(). Both toplines are reported per metric for the deployed winner, so the summary can show, for example, MASE and sMAPE, each as validation and test.

Details: the test window does not move with the CV design

The test window is always each series' final horizon observations, forecast from the same cutoff, whatever cv_n_windows or cv_step_size is. So two runs that differ only in their CV design are scored on the same rows, and their test numbers are comparable. When they differ, the design chose and tuned a different model (more windows reach further back, so older regimes weigh on selection and on the tree backend's search). Two caveats:

A series too short for the larger grid (horizon + step × (cv_n_windows − 1) plus its minimum training length) drops out of cross-validation entirely, and with it out of the test. The run records it, and the test then averages over fewer series.
A change the CV never sees leaves both numbers identical to the last digit: for example, new values for a known-future driver that exist only in the forecast horizon, which no CV window reaches.

Signal check (zero-shot Chronos)¶

Agent access

Agents reach this check through the MCP tool probe_dataset (kind forecast_signal; compute scope, cached). It is the recommended check before proposing a forecasting run. See the ParleML MCP docs.

Before a full run you can ask a cheaper question: does a foundation model beat a naive baseline here at all? parleml_ml4.forecasting.foundation_signal.forecast_signal_check answers it with one zero-shot Chronos backtest over each series' final horizon steps, with no training and no model search. It scores MASE against a baseline on the same held-out window. Per series, the baseline is the better of SeasonalNaive and a random walk with drift, so a trending panel cannot earn a high skill only by beating SeasonalNaive.

from parleml_ml4.forecasting.foundation_signal import forecast_signal_check

res = forecast_signal_check(df, horizon=7, season_length=7)   # df: long [unique_id, ds, y]
res.verdict        # "none" | "weak" | "moderate" | "strong" | "suspicious"
res.model_score    # aggregate Chronos MASE                       (lower is better)
res.baseline_score # aggregate MASE of the better baseline per series
res.skill          # 1 - model/baseline: baseline error removed (0 = no better)
res.skill_ci       # 95% bootstrap interval on the skill, resampling series
res.predictions    # per-series backtest curves (history / actual / forecast / baseline)

The verdict grades the skill, the fraction of the baseline error Chronos removed (1 − model_score / baseline_score):

Verdict	Skill
`suspicious`	`≥ 0.98`: trivially predictable, or possible leakage
`strong`	`≥ 0.50`
`moderate`	`≥ 0.20`
`weak`	`≥ 0.05`
`none`	below `0.05`

A strong or moderate verdict whose bootstrap interval includes zero is downgraded to weak, with a caveat. res.worse_than_baseline is True when Chronos lost to the baseline; that is still none, but it means "a zero-shot model lost to a naive rule", where a stats or tree run is most likely to help.

The check backtests the longest max_series series (default 25) and probes with chronos2_small by default. It needs the forecasting-autogluon extra; without it, it raises PreflightError.

Details: defaults, and the machinery the check reuses

horizon and season_length default from the data when omitted:

Season: a convincing detected period, else the frequency's textbook cycle, or 1 if that cycle tested absent.
Horizon: 2×season, floored at 8 so the measurement means something, capped by a quarter of the median history and by the engine's native prediction length. When the history is too short for the floor, the cap wins and the result carries a caveat.

The check is the forecasting parallel of the tabular parleml_ml4.foundation_signal (TabICL) probe and reuses the real fit machinery: the same ForecastPreprocessor for frequency and calendar, the shared CV grid (a single window of length horizon), FoundationBackend with enable_ensemble=False, the statsforecast backend for the baselines, and compute_metrics.

When there is no regular grid¶

The check always answers. A panel whose timestamps have no regular spacing (an event log, an entity table exported with a created_at) is scored on an ordinal axis: observation order stands in for the calendar. result.grid reports which axis ran ("calendar" or "ordinal"), and result.freq is None in ordinal mode.

This is limited to the diagnostic. A fit produces a deployable model whose forecasts carry real dates, and dating those on a synthetic grid would put false dates into the result frame, exports and MLflow. So Forecaster raises on an irregular panel, pointing at resample_to, while the check degrades to an answer.

Two things to know about an ordinal verdict:

It answers "can the next horizon observations be predicted?", not "the next horizon days". season_length resolves to 1. At m = 1 SeasonalNaive is the random-walk Naive baseline, and MASE is its standard non-seasonal form.
Real calendar seasonality is invisible on that axis. Irregularly sampled data with a genuine weekly cycle can read none. A weak ordinal verdict is not proof there is no signal: set resample_to and re-run to check.

Prefer a rollup when you have one; ordinal is the fallback for when you do not.

Details: horizons past the engine's native prediction length

Chronos-Bolt emits 64 steps in one forward pass and Chronos-2 emits 1024. Beyond that the model rolls over autoregressively, feeding its own predictions back as context. Accuracy decays with distance, and each extra block costs substantially more compute. AutoGluon does not surface this at runtime (the underlying warning is filtered and the rest is logged at debug level), so ml4 detects it ahead of the call.

The auto horizon is clamped to the engine's limit. A horizon you set is never rewritten: it earns a caveat on the check and a failures["_horizon_exceeds_native"] note on a fit. Two Signal Checks of one dataset on different engines can therefore resolve different horizons, which is why the payload always reports the horizon that ran.

Details: one season resolver, one confidence threshold

parleml_ml4.forecasting.season.resolve_season_length is the single authority for season_length: your value → the strongest detected period whose strength reaches SEASON_STRENGTH_THRESHOLD (0.30) → the frequency's textbook cycle, or 1 when the profiler tested that cycle and found no seasonality. The Forecaster, the router, the tree backend's lag / EWMA / rolling plan and this Signal Check all call it.

It is a public, lightweight module. Anything outside ml4 that wants to show the season a fit would pick (a profiling endpoint, a launch wizard) can import it instead of re-deriving the rule. The threshold matters: a period can be statistically significant yet weak. The season is the MASE denominator, the SeasonalNaive baseline the weak-series check is defined against, and every routed seasonal order. So the check's season is the number a later fit will use.

The app serves the check at POST /api/datasets/{id}/forecast-signal-check (with a cached GET), next to the tabular signal-check route. The Dataset detail page's Forecast signal tab shows the verdict, the MASE-vs-baseline metric strip and the backtest prediction graph.

The API surface¶

model = Forecaster(config).fit(X, mlops=None)   # X: DataFrame or SeriesFrame
future = model.predict(h=None, X_future=None, level=None)
future = model.fit_predict(X, h=None)           # fit then predict in one call
cv     = model.cross_validate(X)                 # rolling-origin CV frame, no refit
result = model.result_                           # ForecastResult

Method	Returns
`fit(X, *, mlops=)`	`self`. Optional `MLOpsRunSettings` logs an MLflow pyfunc model.
`predict(h, *, X_future, level)`	Long forecast frame with interval columns.
`fit_predict(X, *, h)`	Convenience wrapper.
`cross_validate(X)`	Rolling-origin CV table for `config.backend`: the same candidates, backend object and CV pass `fit` would run (including the one-window fallback on a panel too short for `cv_n_windows`), on a throwaway preprocessing of `X`. Your fitted state is untouched. A CV failure raises here instead of being recorded. Interval columns only on the `stats` backend; the point-only backends return point columns, as they do inside `fit`.

Reading the result¶

ForecastResult (on model.result_) carries:

Attribute	Meaning
`forecast`	Long-format horizon forecast (Nixtla shape).
`winning_model_per_series` / `winning_backend_per_series`	The selected model and backend per `unique_id`.
`cv_scores` / `cv_table`	Per-candidate CV selection metrics (most recent window held out) and the raw CV predictions (all windows).
`validation_scores`	The deployed winner scored on the selection windows: the card's validation number, directly comparable to `test_scores`. Empty when `cv_n_windows < 2`.
`test_scores`	Honest held-out test: each series' winner scored on its most recent window, excluded from selection. Empty when `cv_n_windows < 2`.
`reconciled_forecast`	Coherent hierarchical forecast (when `hierarchy_cols` is set).
`weak_series`	Per-series warnings: winners that cannot be trusted (fabricated targets, an undefined MASE, or one worse than seasonal-naive). A `tuple[WeakSeries]` (`unique_id`, `reason`, `mase`), computed for every backend. See Honest warnings.
`intervals`	One `IntervalLevelReport` per requested confidence level, recorded when the bands were made: `available` (the forecast carries the level); `calibrated` (`True` / `False` for conformal bands; `False` names the `uncalibrated_series` whose calibration set held fewer than `scores_needed` independent scores; `None` when no band at that level is conformal); `scores_needed` (9 for 80%, 39 for 95%); `test_coverage` (the winners' band coverage on the held-out window); and `calibration_proxies` / `calibration_basis` (`"proxy"` when the residuals came from a stand-in for the deployed model, `"tta"` or `"untuned"`, so the coverage is approximate; `"deployed_model"` otherwise; `None` with `calibrated`). The `_intervals_uncalibrated` note says how to fix a shortfall.
`failures`	Raw breadcrumb dict holding both kinds below. Read it through the two views rather than counting it: `len(failures)` mixes notes with failures (and counts stages, never series).
`failed_stages`	The stages that genuinely failed (bare keys named after the backend: `<backend>_cv` and `<backend>_cv_retry`, such as `stats_cv` or `tree_cv`, plus `<backend>_selection`, `stats_race_collapsed` and `predict`). Empty on a healthy run.
`notes`	Informational breadcrumbs (`_`-prefixed: SeasonalNaive wins, CV-skipped or too-short series, a univariate backend ignoring covariates, budget notes). Routinely present on healthy runs.

Helpers: result.winner_counts() tallies the deployed model mix, result.topline_metric() returns mean CV selection scores per model (biased), and result.topline_test() returns the honest held-out test score per metric (the number to trust).

Persisting a run¶

Pass outputs_dir= to the constructor to write a run layout. fit then writes events.ndjson, manifest.json and ml4.log, honours a CANCEL sentinel, and persists a deployable FittedForecastPipeline under <outputs_dir>/deploy/forecaster/.

model = Forecaster(cfg, outputs_dir="runs/demand").fit(train_df)

See Deployment and MLOps for serving and tracking, and the Forecaster reference for the full API.

Next: Forecasting example · Deployment & export