Fork, don't restore

Picture a git where branch does not exist. Every branch is a full clone, the whole repo pulled from the remote, every time, and every new commit invalidates every clone. Nobody would accept that for code. It is the default for RL environments.

To improve a policy you run rollouts, and every rollout needs its own isolated place to act: a shell, a repo, a browser, a served model. A GRPO step wants a whole group of them branched off the same starting state. A real run wants thousands live at once, created in bursts at the top of every step. And the environment sits on the training critical path, so a synchronous step does not finish until the slowest one is ready. Every millisecond spent booting or restoring an environment is a millisecond your GPUs, the 99% of the bill, spend idle.

We think the answer is not to schedule sandboxes faster, but to stop rebuilding the environment at all. This post is how we got there.

The idea: warm one, fork the rest

The core trick has a name, the zygote, and it is not ours. Instead of booting a fresh process every time you need one, you boot a single process to a warm, fully-initialized state, then fork every new instance off it. Each child inherits that warm state for free and diverges copy-on-write from there. It is the cheapest known way to get a ready-to-work process: don't boot one, copy a warm one.

The most familiar example is in your pocket. Android opens an app the instant you tap it by forking that app off a process it keeps pre-warmed in the background, one literally named the zygote, instead of starting it from cold.

We first reached for the same trick somewhere far less forgiving than a phone: cutting cold-boot time on Mercedes-Benz in-car systems, where a screen that comes up a second late is not a slow demo, it is a defect that ships in a car. Warm one thing, fork the rest, and the boot budget comes back under control.

RL post-training's sandbox fleet turned out to be the same shape of problem: thousands of near-identical environments, all needing the same warm start state, all wanted at once. The zygote answer fit directly, with one hard new constraint. These environments run untrusted, model-generated code across different tenants and different tasks, so a shared process space is out of the question. Every fork has to be hardware-isolated.

So we rebuilt the zygote on microVMs: bake an environment once into a warm, snapshotted VM, then fork that VM per rollout, each fork its own kernel behind a KVM boundary. The same idea that launches phone apps and boots cars, rebuilt with the isolation an RL sandbox fleet demands.

The approaches that stall

Three things people reach for first, and where each one breaks at rollout width.

Cold sandbox per rollout

A fresh sandbox each time. Every rollout cold-starts the environment, launch the browser, import torch, check out the repo, warm the caches. That is seconds, per rollout, and the synchronous step waits for the slowest. Across thousands of rollouts and thousands of steps, the fleet spends most of the run booting.

A hot pool of reusable environments

Keep N environments alive and hand one out per session. Better than cold-launching, but the cost structure is unforgiving: you pre-provision for peak concurrency, idle pool capacity burns money, a pool miss falls back to a cold launch and its tail latency, and reused environments need scrubbing between tenants or state bleeds across rollouts.

Snapshot the warm environment, restore per rollout

This is the right instinct, and it is where most stacks stop. Warm one environment to the start state, snapshot it, restore every rollout from the snapshot. It breaks at width because restore is O(size). Restoring a warm multi-GB environment re-materializes its whole memory image, and you pay that per copy: restore it a thousand times and you move terabytes to produce a thousand identical machines. Filesystem-only snapshots are quicker but throw away process memory, so you are back to warming the environment on every restore.

You can place an empty sandbox in half a second. You still cannot make a thousand warm 2 GB environments exist without moving two terabytes to do it. That is a memory problem, not a scheduling problem.

The design: bake once, fork the RAM

Collimate does exactly what the Android zygote did, on microVMs. Two phases.

Bake, to build a warm microVM zygote

You bring an OCI image and declare a ready state: the warm-up that must complete before a rollout can start, launch chromium, import torch; load_weights(), wait on /health, navigate to a start page. We run that warm-up once inside a KVM microVM and snapshot the machine at that exact point. The snapshot's RAM contains a live, fully-initialized environment. That is the zygote.

Fork, copy-on-write, per rollout

A rollout is a copy-on-write fork of the zygote. Nothing is copied until the child writes to a page, so the child inherits the warm heap, the launched browser, the loaded model, the whole running machine, and diverges independently. Fork latency is ~2.4 ms at p50 and the marginal resident cost is a few megabytes, independent of the nominal VM size. It is git branch, not git clone, for the entire machine, memory included.

Because each fork is its own microVM, the isolation the Android zygote could not give us comes for free: a separate guest kernel, a per-fork network namespace, a hardware KVM boundary. One tenant's model-generated code cannot see another's, and a fork that corrupts itself takes nothing with it.

Why a thousand task images don't cost a thousand bases

RL runs are image-heavy. A SWE-style run has thousands of task images, one per repo, that mostly share a base. Rebuild and re-pull all of them naively and you drown in disk, pull time, and resident memory.

So the environment's filesystem does not live inside the snapshot as one flat blob. It sits in a custom, deduplicating, copy-on-write filesystem we built: every layer is stored once, and a base shared by a thousand task images is stored once, shipped once, and resident once. The guest reads it in place, with no per-VM copy. We measured a shared base that is hundreds of megabytes on disk showing up as effectively zero additional resident memory for each environment that uses it. The only per-template resident cost is the warm-RAM snapshot itself, which is almost entirely zero pages and staged sparse.

This is also the answer to the CI rebake tax. An environment bakes once into an artifact of deduplicated layers, pullable from any registry like a normal image. A re-pushed tag only rebakes the layer that changed. And a "restore" is not a reinstall, it is a millisecond fork of the ready state.

The numbers, from running it

Everything below is measured. Unless noted, it is on a single 16-vCPU box with nested KVM, the same policy model and identical tasks across engines, only the sandbox engine differs.

Forking and density

• Fork: p50 2.4 ms, p99 5.5 ms over 1,000 iterations. On the 2 GB SWE-bench image, create is 4.5 ms cold and 0.5 ms on a warm-pool hit.
• Resident memory: about 2.3 MB per live sandbox versus about 260 MB for a full microVM per rollout running the same work, roughly 110x less, regardless of the VM's nominal size.
• Density: 2,000 live headless-Chromium environments in 10 GiB of host RAM on one 16-vCPU box.
• Fleet: 100,000 concurrent sessions across 20 nodes at 5,000 each, 100% success.

Sustained, at width

Two 12-hour runs on one 16-vCPU box with the 2 GB SWE-bench image:

• A 1,024-wide GRPO step fans out in p50 1.22 s; 206 steps, about 211,000 sandboxes created, 0 errors.
• A 256-concurrency churn ran 3,448,306 rollouts at 0.0003% error, 12 total, with zero state drift across the whole run.

End-to-end RL, versus another microVM sandbox

E2B, like us, runs each sandbox as a microVM, so the isolation is the same. What differs is the warm start: E2B gives every rollout its own full VM with full resident memory and a roughly one-second warm-resume, where we fork a warm snapshot copy-on-write in milliseconds off shared memory. On matched runs, same 16-vCPU box, same policy model, identical tasks:

Versus a container runtime

Container-based sandbox runtimes make a different trade. Daytona, for example, creates a cold sandbox in about 90 ms and its fork is filesystem-only, with no live-memory checkpoint, so a reset still cold-launches the browser and holds a full-RSS sandbox per session, capped near a hundred concurrent on a tier. We held 256 to 384 warm browsers, copy-on-write shared off a single 2 GB snapshot on one box, each handout about 30 ms. The same guest image runs on either; the difference is entirely in how the fleet holds it.

Web-agent rollouts: forking removes real blocking time

The sharpest case is browser RL, because a reset there means re-navigating to the start state. We forked a warm-resident headless Chromium, snapshotted at a deep page, versus re-navigating and replaying the path, the pooled-browser reset, and the same cost a tree search pays on every backtrack. The forked episode is flat regardless of start depth; the re-navigate episode grows about 24 ms per page replayed. At depth 3, forking eliminates 28% of the episode; at depth 8, 44%. A WebArena episode is about 30 steps, where the replay is seconds and forking removes the large majority of the rollout.

And against memory-snapshot-restore head to head: a fork lands in 0.87 ms with the parent surviving, where restoring the same warm state runs 69 to 106 ms plus a one-time capture, and capturing a 2 GB memory snapshot took 17.6 s. O(1) against O(size), the same asymptotic gap, measured.

Compute your own tax

Whatever the bandwidth, the structure does not move: a restore-based fleet re-copies the environment's full RAM for every rollout, so its pre-warm scales with environment size times rollout width, and it is re-paid on every image push. One line, for any restore-based platform:

We've been running this

We drive this fleet on SWE-bench code-repair, terminal-bench, web-agent rollouts over real headless Chromium, and heavy torch environments. The numbers in this post come from those runs. It is live now as a managed service, and an API key is all it takes to fork a live SWE environment into a rollout group, stand up a hundred live browsers, or bake your own environment to a warm ready state and fork it wide.

There is a second half to this we will write up on its own. Because the fork is this cheap, you can fork an agent mid-episode and branch a rollout group off a running trajectory, which is what makes per-step credit assignment, VinePPO and its relatives, practical on stateful agents. That is its own post.

The conclusion we keep coming back to: for RL post-training, the environment fleet is a memory problem. Boot once, warm once, and share the warm state by forking it, instead of re-materializing it a thousand times a step. That is why the loop stops waiting on the sandbox.

The cost model, for the ML-infra crowd

Per GRPO step, let W = rollout width, M = environment memory footprint, B = restore bandwidth, t_f = per-fork latency, N = concurrent provision slots (nodes times per-node parallelism), K = training iterations (image pushes).

Restore, per rollout. Each sandbox rehydrates its full memory M out of storage:

T_prewarm(restore) = W * M / B      = O(M * W)      # linear in BOTH size and width
marginal_RAM/box   = M                              # full private copy, nothing shared
over K pushes      = K * W * M / B   = O(K * M * W)  # snapshot invalidated on every image push

Fork, per rollout. Forks share pages copy-on-write; no per-sandbox memory copy:

T_prewarm(fork) = W * t_f / N     = O(W),  t_f = O(1) in M   # independent of memory size
marginal_RAM/box = ~0                                        # copy-on-write shared
over K pushes    = ~K * W * t_f / N                          # thin layer diff + cross-template dedup

The takeaway for RL post-training: restore provisioning is O(M·W), re-paid every iteration, so O(K·M·W). Fork provisioning is O(W) forks at O(1) per fork in memory size, with about zero marginal RAM. As environment size and rollout width grow, the gap does not shrink, it widens.