API Reference

Technical reference for FlexTensor's public API. All pages below are auto-generated from source docstrings.

Simplified API

Module-level convenience functions — the recommended starting point for most users.

• DEFAULT_MANAGER_NAME — Name used by get_offload_manager() when no explicit name is provided (value: "default")
• init() — Pre-initialize the tensor manager (optional, called automatically by offload())
• offload() — Patch a model for automatic tensor offloading
• get_offload_manager() — Get or create an offload manager singleton
• set_config() — Set configuration for an offload manager
• offload_block() — Context manager for manual offloading control
• get_gpu_memory_usage() — Get GPU memory usage (inference mode only)
• save_profile() / load_profile() — Profile persistence
• offload_from_profile() — Load a saved profile and offload in one step
• offload_from_state() — Adopt an in-memory saved state
• release() — Release resources and restore model
• collect_offload_timing() / reset_offload_timing() / update_offload_timing() — Inference offload-timing measure window
• request_strategy_replan() / update_state() — Remeasure and rebuild strategy (compile / CUDA graphs)
• StrategyReplanStatus — Passive replan availability and outcome, read from OffloadManager.strategy_replan_status
• OffloadTimingReport / OffloadTimingSnapshot — Timing aggregates and per-pass data
• load_model_from_profile() — Load a model from a saved profile with optimized weight loading

OffloadManager

The OffloadManager class provides full control over the offloading lifecycle. Obtain an instance via get_offload_manager().

Configuration

• OffloadConfig — All offloading parameters with sensible defaults
• load_config() — Load config from files, env vars, and kwargs
• GPUMemoryUsage — GPU memory usage breakdown

Strategies

Offloading strategies for OffloadConfig.load_strategy:

• Strategy — Protocol all strategies implement
• KnapsackStrategy — Dynamic programming optimization (default)
• GreedyStrategy — Largest tensors first
• GlobalOffloadStrategy — Global optimization across layers
• And more...

Advanced

Lower-level components for custom workflows:

• TensorManager — Direct control over discovery, profiling, and inference phases
• TensorBenchmarkMode — Benchmark mode base class
• BenchmarkReplace / PreloadToDevice — Benchmark implementations
• Host pinning — Lower-level pinned-memory helpers and modes