Prime Intellect slide 1
Prime Intellect slide 2
Prime Intellect slide 3
Prime Intellect slide 4
Prime Intellect slide 5
Prime Intellect slide 6
Prime Intellect slide 7
Prime Intellect slide 8
Prime Intellect slide 9
Prime Intellect slide 10

GTC EUROPE - BERLIN - OCTOBER 2026

Post-training open models into computer-use agents

Online RL with prime-rl, and what NVIDIA Dynamo changes in the inference loop

Emrick Sinitambirivoutin, Core Team Research Engineer, H Company

hcompany.ai

TRAINING PIPELINE

FROM AN OPEN MODEL TO AN AGENT

Post training @ H Company

How we specialize models for computer use

REF hcompany.ai/newsroom/holo4 (2026-09-28)

Agentic Task Factory

coding agents build envs + verifiers · ≈10,000 tasks

Open base model

Qwen3.8 27B

Nemotron 3 Nano Omni

SFT

Trained on 127B tokens, from our Agentic task factory
(GUI, API, MCP)

Online RL

Trained on live environments with verifier rewards, GRPO and LoRA adapters

Merge

experts folded back

at equal weight

no further training

Holo4 · Holotron4

desktop · web · mobile

MCP · API · terminal

SFT MIXTURE, SHARE OF 127B TOKENS

45%

Desktop trajectories

14%

Web

12%

MCP + API

Mobile 3%

26%

Grounding · reasoning · tool use

01 / 14

INFERENCE FLEET

Frontend / router

Engine workers, ranks 0…N

routes requests

PRIME-RL

HOW ONLINE RL IS WIRED

prime-rl high level architecture

3 components that run independently

REF prime-rl (github.com/PrimeIntellect-ai/prime-rl)

Orchestrator

Trainer

1 discovery

2 rollout requests

3 admin plane

pause · load adapter · resume

4 weight transfer

5 training batch

02 / 14

INFRASTRUCTURE TODAY

REF SkyPilot · github.com/skypilot-org/skypilot · docs.skypilot.co

1 · sky launch

API server queues

the request

2 · Provision

N pods on Kubernetes (EKS)

each 8×H100 · 128 CPU · 1.9 TB

3 · Bootstrap

Ray cluster + skylet per pod

S3 / EFS mounts · code sync

4 · Launch

one Ray task per pod runs

setup, then run (NODE_RANK)

SKYPILOT CLUSTER · ONE PER RUN

rank 0

Inference

vLLM + router

+ orchestrator

Ray · skylet

rank 1

Trainer

Ray · skylet

rank 2

Trainer

Ray · skylet

rank 3

Trainer

Ray · skylet

rank 4

Trainer

Ray · skylet

FIXED UNTIL THE RUN ENDS

The inference fleet is fixed, which means:

• We cannot adjust inference replicas to the load

• A failing inference replica fails the job

03 / 14

HOW A RUN IS LAUNCHED

Prime-rl @ H Company

How we use Skypilot to orchestrate prime-rl in our infrastructure

INFRASTRUCTURE WITH DYNAMO

REF NVIDIA Dynamo · github.com/ai-dynamo/dynamo · SkyPilot · github.com/skypilot-org/skypilot

SKYPILOT CLUSTER · ONE PER RUN

rank 0

Trainer + orchestrator

Ray · skylet

rank 1

Trainer

Ray · skylet

rank 2

Trainer

Ray · skylet

rank 3

Trainer

Ray · skylet

No inference pods: rollouts go to the shared fleet.

KUBERNETES · SHARED INFERENCE CLUSTER

Dynamo frontend

OpenAI API · Rust chat processor · KV-aware router

/v1/rl/workers discovery

Worker 0

Worker 1

+ worker

joins or leaves

mid-run, loads the

current adapter

EFS

shared adapters

etcd

worker discovery

ModelExpress

fast boot over RDMA

rollouts (chat, MITO)

discovery

load adapter, each

adapter files

04 / 14

WHAT CHANGES

Prime-rl @ H Company with Dynamo

How we use our shared inference infrastructure for RL

OPTIMIZATIONS

THE STARTING POINT

Where the time went

Inconsistent prefixes slowed training and inference

REF prime-rl (github.com/PrimeIntellect-ai/prime-rl) · means over training steps 3–50 of the first run (3-screenshot window, 2 vLLM servers)

05 / 14

ONE TRAINING STEP: 285 S

forward + backward 217 s

waiting for rollouts 57 s

The trainer computes 80% of the step. Two things made it slow:

01

Every turn is its own training sample

About 24 turns per rollout, each trained alone: the trainer re-reads the whole history every turn.

TOKENS THE TRAINER READS PER ROLLOUT

prompt, re-read: 98%

completion, trained: 2%

02

The prompt changes at every turn

The 3-screenshot window drops an image each turn: turns can't share a sample, and the cache is invalidated.

PREFIX-CACHE HIT ON THE INFERENCE SERVERS

0.73

Next: an append-only prompt (folded history), then one sample for consecutive turns (turn merging).

MULTIMODAL RL

FIXING THE TRAINER · 1 OF 2

Folded screenshot history

Invalidate the prefix cache once every 5 turns, not every turn

REF Folded history as in Qwen-CUA (keeps 20 screenshots, folds 10); ours keeps 10, folds 5 · vLLM automatic prefix caching

06 / 14

WHAT THE PROMPT OF TURN 7 REUSES FROM TURN 6

Rolling window

last 3 screenshots

1

2

3

4

5

6

7

turn 4 lost its screenshot → recompute KV cache from there (4 turns)

Folded history

up to 10, then drop 5

1

2

3

4

5

6

7

nothing changed: only the new turn is computed

CACHE INVALIDATIONS OVER THE FIRST 20 TURNS

Rolling window

17 invalidations

Folded history

2 invalidations

1

4

11

16

20

prompt extends the last one

prompt changed: recompute

Longer prompts, but the cache survives 4 turns out of 5.

W&B · PREFIX-CACHE HIT PER STEP

PREFIX-CACHE HIT

0.73 → 0.83

MULTIMODAL RL

FIXING THE TRAINER · 2 OF 2

Turn merging

A 22-turn rollout trains as 4 samples instead of 22

REF prime-rl (github.com/PrimeIntellect-ai/prime-rl) · token bars from one recorded rollout, first 10 turns

07 / 14

Each turn as its own sample

turn 1

turn 2

turn 3

turn 4

turn 5

turn 6

turn 7

turn 8

turn 9

turn 10

120,100 tokens through the trainer

prompt: read, no loss

The same turns as one merged sample

16,590 tokens, 7.2× fewer

completion: trained

A turn joins the previous sample only if its prompt extends it exactly: we fixed the chat template's re-rendering of past reasoning, in our stack and in Dynamo's frontend.

W&B · TRAINING SAMPLES PER ROLLOUT

TOKENS PER TRAINING STEP

22.6M → 4.7M

FORWARD + BACKWARD

259 → 89 s

Now inference-bound: the trainer waits 144 s per step for rollouts.

NVIDIA DYNAMO

FIXING INFERENCE · ROUTING

Routing multi-turn rollouts

Pin each rollout to one worker: prefix-cache hit 0.62 → 0.81

REF NVIDIA Dynamo KV router (github.com/ai-dynamo/dynamo), DYN_ROUTER_SESSION_AFFINITY_MODE=hard · figure: illustration

08 / 14

session-aware:

X-Session-ID → A

KV-aware:

lighter load wins → B

Turn 9 of rollout R

Dynamo router

Worker A

KV CACHE

R, turns 1–8

other rollouts

RUNNING REQUESTS

24

prefix reused: only turn 9 is prefilled

Worker B

KV CACHE

other rollouts

free

RUNNING REQUESTS

12

prefix recomputed: turns 1–8 prefilled again

STEPS 3–50

CACHE HIT

STEP

GPU-hours / 100

prime-rl vLLM, 4 servers

0.83

192 s

213

Dynamo, KV-aware

0.62

192 s

214

Dynamo, session-aware

0.81

168 s

188 (−12%)

W&B · PREFIX-CACHE HIT PER STEP

AUTOSCALING

FIXING INFERENCE · AN ELASTIC FLEET

Scaling on the trainer's wait

Load-based auto scaling can't see the trainer waiting

REF NVIDIA Dynamo (github.com/ai-dynamo/dynamo) · Kubernetes Horizontal Pod Autoscaler · run with the load-based autoscaler, steps 3–100

09 / 14

The problem with scaling on load

What the autoscaler sees

20–25 requests per worker, under its target of 30: it never scales up.

REQUESTS PER WORKER

target 30

What the trainer sees

It waits 51 s of every 153 s step for its slowest rollouts.

ONE STEP: 153 S

51 s

waiting for rollouts

Solving it with a custom controller

INPUTS, EVERY 5 MIN

Trainer

step time, wait

Workers

load, latency

Rollouts

time in model calls

OUTPUT

Autoscaler

min = max = N

Cost-aware controller

1

Fit latency vs. requests per worker

2

Predict the step time for each fleet size

3

Price each size in GPU-seconds per step

4

Pick the fastest within 3% of the cheapest

Dynamo fleet

N vLLM workers

new metrics

AUTOSCALING

FIXING INFERENCE · AN ELASTIC FLEET

The controller follows the trainer

The HPA holds its fleet; the controller scales with the wait

REF NVIDIA Dynamo (github.com/ai-dynamo/dynamo) · Kubernetes Horizontal Pod Autoscaler · live runs, training steps 3–100

10 / 14

HPA on load

target 30 requests per worker

4

8

workers

trainer wait

WORKERS, MEAN

4.1

Controller

trainer-aware, cost-aware, max 10 workers

4

8

workers

trainer wait

WORKERS, MEAN

7.7

step 3

25

50

75

100

NVIDIA DYNAMO

FIXING INFERENCE · FAST BOOTS

ModelExpress

A new worker serves in 2.6 minutes instead of 11.1 minutes

REF NVIDIA ModelExpress (github.com/ai-dynamo/modelexpress) · NIXL (github.com/ai-dynamo/nixl) · vLLM (github.com/vllm-project/vllm)

11 / 14

image pull

process start, registration

weights

engine init: compile, profile, graphs

ModelExpress + warm cache

67

85

2.6 min

Weights from S3, no cache

59

190

172

7.0 min

… and image not on the node

242

59

190

172

11.1 min

DECISION → SERVING

2–3 min vs 6–8

FLEET REACHES 8 WORKERS

step 28 vs 34

STEP TIME, SAME GPU-HOURS

119 s vs 125

RESULTS

SAME CODE, 100 STEPS EACH

Results

Faster training (×1.24) at lower cost (5% less)

REF prime-rl · NVIDIA Dynamo · ModelExpress · trainer on 4 nodes × 8 H100 in every setup · GPU-hours include the trainer's idle time

12 / 14

SETUP

INFERENCE GPUS

STEP TIME (speedups)

TRAINER WAIT

100 STEPS

GPU-Hours / 100 STEPS (savings)

prime-rl

4 vLLM servers (TP2), fixed

8

147 s

51 s

4.1 h

164

Dynamo + controller

trainer-aware controller sizes the fleet

13.1

125 s ×1.18

20 s

3.4 h

157 −4%

Dynamo + controller + ModelExpress

fast worker boots

14.8

119 s ×1.24

14 s

3.3 h

155 −5%

RESULTS

THE SAME RUNS IN W&B

Training step vs wall-clock

Step 100 at 3.3h vs 4.1h for prime-rl

REF Weights & Biases · minutes since each trainer started; the results table times steps 2–100

13 / 14

W&B · TRAINING STEP VS MINUTES SINCE START

TAKEAWAYS

WHAT TO REMEMBER

REF hcompany.ai · github.com/PrimeIntellect-ai/prime-rl · github.com/ai-dynamo/dynamo

Thank you

14 / 14

Dynamo in real RL setup: one trainer, rollouts on three continents

A production customer's RL setup, redrawn. Each number is one of the next three demos.

1

Global routing

One KV-aware router
across clusters and regions

2

Fault tolerance

Dead replica → reroute session
trajectory survives

3

Delta weight transfer

>99% fewer bytes over the WAN
3–4 s inference pause per update

Trainer one cluster · US Δ weights Weight sync via object storage Rollout EUUS WestUS EastAPAC scored rollouts back to the trainer 1Global routing 2Fault tolerance 3Delta weight transfer
01 · Global routingSimulation

1Global routing: One global, KV-aware router instead of a static split

AOne router, one global pool

Dynamo router EUUS WUS EAPAC older

All four datacenters act as one pool. No fixed share of the batch per site.

BEach turn goes to the soonest finish

Where should this turn run? W3W9APAC cache hitidle, no cacheslow GPU ✓ picked queue waitrun turn

A KV-cache hit makes the turn cheap. Slow GPUs cost more, so long rollouts leave them.

CEvery site finishes together

Fixed splitDynamo APAC idle step donestep done ~⅓ shorter RL step

No datacenter sits idle waiting on the slowest site, so training starts sooner.

t = 0
100 concurrent multi-turn sessions · square = worker, shade = busy slots, number = queue · APAC = older GPUs
02 · Fault tolerance · fills Prime slide 06Simulation

2Fault tolerance: a worker dies, the rollout keeps going

AHistory lives outside the worker

sandbox proxy every token logged GPUcache

If the GPU dies, the session’s history survives in the proxy.

BSessions move, not restart

reroute one session’s progress kept: all earlier turnsredo 1 turn

Sent to the least-loaded healthy worker. Only the current turn is redone.

CReplacement catches up fast

Full weights (2 TB) cold start ~25 min Checkpoint + Δ replay ~5 min then sessions rebalance onto it

Loads a checkpoint plus small Δ updates, not the full 2 TB model.

t = 0
48 trajectories × 1–5 h · 8 workers · same batch in both lanes · illustrative
03 · Delta weight transfer · fills Prime slide 05 “path TBC”Simulation

3Delta weight sync: one trainer, rollouts on three continents

ATrainer publishes Δ

version N ⊕ N+1 (bit-by-bit XOR) 0 0 0 1 0 0 0 0 0 1 0 0 0 0 full weights 2 TB Δ after zstd ~20 GB · >99% smaller

Mostly zeros → >99% smaller

→

BVersioned in object storage

ckpt ΔΔΔΔ … full checkpoint + chain of deltas

One upload per step

→

CPeers fan out in-region

one pull across the WAN one region W0 W1W2W3 NIXL P2P over RDMA

Peers copy over NIXL, not the WAN

→

DApply at a safe point

rollout generation policy Npolicy N+1 3–4 s pause

Swap weights, clear KV cache

Same update, two ways. Trainer makes a new step every 2 min (1T model).
❗Demo values are illustrative
BF16 weights · 3 remote regions · analytic model, illustrative
What to measure together · maps to Prime slide 10, Experiment ASimulated, directional

Summarizing additional Dynamo RL value prop

1Global routing
Fixed splitDynamo APACidle

Every datacenter finishes together

2Fault tolerance
Without Dynamo ✕ whole rollout lost · restart at turn 0 With Dynamo all earlier turns keptredo 1 turn

A failed worker costs one turn, not the rollout

3Delta weight transfer
Full weights · 2 TB per update ~11 steps behind trainer (off-policy) Δ weights · ~20 GB per update ≤1 step behind trainer (on-policy)

Rollouts stay on-policy

★github.com/ai-dynamo/dynamo
Figures on this slide come from the simulation unless attributed to a production customer
All slidesclick a slide to jump · G or Esc to close