Self-benchmark, then freeze
Loading protocol…
FPM Gym · Evaluation protocols
One dataset. Five ways to learn.
Compare interpolation, linear regression, spline regression, GBDT, hybrid calibration, and the causal selector under the same training and evaluation protocol.
Measurements come from the pinned Hugging Face dataset in the evaluation snapshot. Strict C starts empty and admits one scored observation every five seconds, including warmup.
Loading protocol…
A
Waiting for split results.
Loading scenario results…
Compare methods
| Model / configuration | GPU / framework |
|---|
Hatched = unsupported combinationBlank = unavailable · Hover or select for the reasonErrors and evaluated zero-coverage results stay visible.Counts show successful / measured test observations.
All scenarios · Selected configuration
| Scenario | InterpolationKV warmup on | InterpolationKV warmup off | Linear | Spline | GBDT | Hybrid calibrationfrozen GBDT | GBDT selectorCausal expert selection |
|---|
Read each result in its training context
The scenario is the third dimension: switch scenarios to compare methods across configurations, then select a configuration to see every scenario together.
Split the collected dataset
Loading protocol…
Learn from the stream
Loading protocol…
Bootstrap, then adapt
Loading protocol…
Train on the past, adapt to the future
Loading protocol…
These scenarios reuse the same recorded measurement corpus to emulate offline training and online adaptation; they do not run a live deployment. A scenario does not establish a named workload or production traffic distribution.
Applicability, coverage, and timing
Interpolation uses a collected library and is evaluated only in A. KV warmup on/off select different collected libraries.
For A and A+C, learned predictors use AISim’s verified original-A fallback for missing mixed-workload stores: a whole-forward interpolation prior, with calibration and online expert routing where applicable. B, C, and B+C receive no original-A prior. Unsupported interpolation remains unavailable.
Direct interpolation operates on the measured self-benchmark table. SOL interpolation uses model-based normalization before interpolation. Successful cells show the recorded choice when available; these results do not all use the same interpolation calculation. Select a result to inspect its timing-only, text-only, and verified legacy metadata recovery settings. Results without these fields remain unclassified.
MAPE includes successful predictions only. Read it with prediction coverage and the test count: cold starts, missing inputs, and prediction failures remain in the measured denominator. Different scenarios use different training and test populations; lower MAPE alone does not establish a better method.
Training time covers initialization and offline fit. C has no offline fit; any cold-start fitting happens inside update calls and contributes to update cost. Preparation counts describe selected inputs, not successfully fitted samples. Self-benchmark regression repeats artifact rows for each worker; interpolation counts artifact rows. Unavailable results can retain nominal selections even when preparation did not run. Prediction and update costs are mean host wall times per invocation in the evaluator, not GPU serving latency. Cost metrics describe the entire scenario, independent of the workload filter. Missing timings remain unavailable.
Each evaluation pins its code and dataset revisions in the snapshot above. Results are loaded from its published scenario artifact; no scores are estimated in the browser. These are single-run measurements; timing can vary between repeated runs.