/backtestsadminBacktest: fit the release of a design, a run or an inline release on a measured series BEFORE the cut and forecast after it; scored only out of sample at each horizon (MAPE, sMAPE, MAE, pinball, cumulative error) beside naive baselines (last period carried forward, no further change, seasonal-naive when the cadence has a season), with a plain verdict on whether the model beat each. A signal and an outcome make it a calibration pair. All faults come back at once in faults [{path, message}].
Body
subject | what the series measures |
series | {values: [8..1000 numbers >= 0], kind: cumulative|per_period, cadence: hour|day|week|month|quarter, start?: YYYY-MM-DD, measure?: adoption (new adopters per period, default) | attention (level per period)} |
cut | points before the cut (4..n-1), or a date with series.start |
horizons | days, 1..8 (default [7, 30, 90]) |
design | a design key (its release seeds the fit) |
run | or a run id |
release | or {channels, imitation, fatigue, fatigue_shape} |
seed | whole number (1): the same seed, the same fit |
season | periods per season (default day 7, week 52, month 12, quarter 4) |
margin | 0..1 (0.05): how much lower the MAE must be to count as beating |
signal | {run, metric: mean|would_act|have_it_end|ever_had_it_end, measure?} (debiased by the run's measured prior) or {value, metric?} |
outcome | number observed (default: the total after the cut to the longest horizon) |
