AISimulate - FPM Predictors

Regression average MAPE

—

FPM (KV warmup on) average MAPE

—

FPM (KV warmup off) average MAPE

—

Learned (fpm5) average MAPE

—

How the predictors work

One online learner, two measured-library predictors, and one offline-trained model.

Regression

Learns from preceding observations per worker. Predict first, then tune on the observed latency.

Input: Previous worker observations
Mode: fpm_regression

Python example

config = replace(
    base_config,
    estimation_mode="fpm_regression",
    fallback_policy="deny",
)
model = RustForwardPassPerfModel.best_available(config)
latency_ms = model.estimate_forward_pass_time_ms(workload)
model.tune_with_fpms(observations)

FPM (KV warmup on)

Predicts from whole-forward measurements using interpolation and supported analytical scaling.

Input: Library collected with KV warmup
Mode: fpm_interpolation

Python example

config = replace(
    base_config,
    estimation_mode="fpm_interpolation",
    fallback_policy="deny",
    systems_paths=(warmup_library_root,),
)
model = RustForwardPassPerfModel.best_available(config)
latency_ms = model.estimate_forward_pass_time_ms(workload)

FPM (KV warmup off)

Uses the same interpolation and analytical scaling with measurements collected without KV warmup.

Input: Library collected without KV warmup
Mode: fpm_interpolation

Python example

config = replace(
    base_config,
    estimation_mode="fpm_interpolation",
    fallback_policy="deny",
    systems_paths=(no_warmup_library_root,),
)
model = RustForwardPassPerfModel.best_available(config)
latency_ms = model.estimate_forward_pass_time_ms(workload)

Learned (fpm5)

Gradient-boosted trees (150 trees, at most 15 leaves each) over the five FPM v1 counts and sums, fitted once. Within each workload kind every other observation trains the model and the rest are scored: this measures interpolation within the measured range, and coverage is about half by design.

Input: Alternate worker observations
Mode: learned

Python example

artifact = fpm_learned.train(
    training_iterations,
    worker_type="decode",
    features=fpm_learned.FEATURE_PRESETS["fpm5"],
    max_iter=150,
    max_leaf_nodes=15,
    learning_rate=0.2,
)
model = RustForwardPassPerfModel.from_learned(artifact)
latency_ms = model.estimate_forward_pass_time_ms(workload)

KV warmup describes how the input library was collected. It is not a prediction-time Boolean.

Examples use replace from dataclasses and RustForwardPassPerfModel from aisimulate_core.sdk. Supply a ForwardPassPerfModelConfig as base_config with matching model, hardware, backend, topology, and worker role; use a separate regression model per worker.

See the configuration API and Gym regression settings for the evaluated regression policy, which differs from SDK defaults.

Summary cards average configuration MAPEs equally. Model table rows are weighted by successful prediction count. Learned (fpm5) is scored on a different subset (alternate observations of each workload kind), so compare it with the other columns as an indication, not a ranking. The fpm_learned example imports from aisimulate_core.sdk.

MAPE vs silicon — successful forward-pass predictions

Accuracy grouped by model and configuration. Activate a column heading to sort model groups. Expand a model to inspect configurations.

Loading accuracy matrix…

* Serving layout is unknown; Agg is shown by default. N/A means not applicable.