Cache-Billed Versus Capacity-Billed Filesystems for AI Agent Workloads

For isolated per-tenant storage in multi-tenant AI agent applications, the architecture is a persistent workspace per tenant, session-isolated execution, and tenant-scoped credentials enforced at the storage or network boundary. The economic fit is active-data billing.

How prompt caching inverted the economics of agent inference

The counterintuitive finding was that the agent harness loading the most context paid the smallest bill. More input did not necessarily mean more expense. A team that develops agents found that KV cache hit rate mattered more to the cost of production agents than any other metric, ahead of token count or model choice.

Every agent loop re-sends a conversation prefix that holds the system prompt, tool schemas, and conversation history and grows a little each turn. When there is no cache hit, the model processes that prefix again. One measured production profile showed roughly 267 input tokens read per output token generated, which helps explain why prefill dominates the bill.

That changes the prompt-design rules. Stable context should be front-loaded and pinned, with only the fresh delta appended. A large, stable prompt can be cheaper than a smaller prompt that keeps changing.

Why cache hit rate is fragile when the cache lives only on the provider's servers

With provider-managed caching, that discount depends on rules the agent operator cannot set. A provider’s TTL determines how long cached context remains available, so a pause between sessions can outlast the opportunity to reuse it. A well-preserved prompt is not a reservation on the provider’s servers. The discount can disappear even when the application behaves consistently.

Server load creates a separate failure mode. KV cache blocks occupy HBM, competing with model weights and other active decoding sessions. Under queue pressure, blocks from paused or earlier sessions can be evicted. The inference cost analysis in arXiv 2605.26297 describes how effective caching sharply reduces the append ratio, while cache thrashing restores prefill as the dominant expense.

Teams that cannot control the inference engine thus need to control what goes into context and what comes out. Persistent workspaces preserve the files and history needed to reconstruct a stable prefix, although they cannot prevent provider eviction. Archil provides persistent disks that survive agent restarts and can branch in milliseconds, making it one storage option for maintaining reusable context independently of provider TTL policies.

What storage billing models agents actually encounter

For an operator using capacity-billed storage, provisioning a workspace means paying for its reserved size whether the agent accesses it or not. Block and flash-backed storage for container-orchestrated stacks, including Simplyblock, offers predictable low-latency behavior for mixed read/write memory patterns, but typically provisions capacity. Predictability comes with an idle-space bill.

Active-data-billed, or cache-billed, storage instead charges for data that is actively resident. Cold data costs nothing or nearly nothing at that layer. This fits an agent that works in bursts, pauses, and later resumes without needing its entire workspace resident throughout the gap. The billing unit follows activity rather than the largest volume the operator might need.

Object storage adds another cost layer. S3, GCS, R2, and Azure Blob charge for stored bytes and request operations, with egress a significant expense on most platforms. Cloudflare R2 eliminates egress charges, changing the calculation for repeated reads. Filesystem-over-object-storage layers let agents use POSIX semantics with buckets they already have without a separately provisioned volume, but the bucket’s storage charges still apply alongside active-data billing at the filesystem layer.

How capacity billing penalizes the prompt engineering strategy caching requires

Inference rewards keeping a large prefix stable, while a capacity-billed volume charges for the space supporting that accumulated context even when the agent is stopped. Bursty work widens the gap. The workspace must have room for peak demand, and the bill keeps running between sessions.

Ephemeral sandboxes avoid retaining the environment by wiping it when the session ends. On restart, the application must rebuild packages, files, and conversation history, and it cannot get the same cache reuse if the prefix has changed or is no longer available. Saving on retained workspace can push expense back into prefill.

Compaction makes this tension harder to manage. Summarizing or rewriting the stable prefix creates a cache cold start, while preserving workspace state can avoid unnecessary reconstruction across restarts. Even with persistence, the model’s context window has a limit, and a provider cache can expire.

Under capacity billing, operators must compare what they save on inference with what they pay to keep or reserve storage. Active-data billing removes the charge for idle provisioned headroom from that tradeoff.

Persistent, isolated sandboxes as the storage layer that makes the economics work

Microsoft disclosed CVE-2026-25592 and CVE-2026-26030 in May 2026, showing prompt injection in agent frameworks achieving host-level remote code execution. OpenAI’s o1 system card documented a model exploiting a misconfigured Docker API on an evaluation host. Those incidents make the execution boundary consequential. An agent that runs untrusted code needs isolation that limits its reach beyond the sandbox.

Persistence must exist inside that security design. A persistent sandbox retains installed packages, files, and conversation history across stops and restarts, giving the agent a reusable environment. Without isolation, state that lasts also leaves a security problem that lasts.

Northflank Cloud uses microVMs for CPU sandboxes and gVisor isolation for GPU sandboxes, showing that the approaches can serve different workloads within one platform. For agents installing packages or contacting external services, the appropriate configuration scopes egress to approved destinations and injects secrets at the network boundary. Pausing compute then becomes a separate question from retaining files. A durable disk can keep the workspace when idle compute stops, and active-data billing can spare the operator the cost of a fully provisioned idle volume.

Forking agent workspaces to run parallel attempts without paying for full copies

Parallel exploration starts several agent runs from a shared checkpoint, lets them try divergent strategies, evaluates the outputs, and keeps the winner. The discarded attempts are part of the method. For this pattern to work interactively, each branch needs the prepared environment without waiting for a full disk copy or reinstalling dependencies.

Copy-on-write snapshots or a write-ahead log allow branching and rollback without duplicating the whole disk at fork time. Billing on active written data then avoids treating every branch as another fully allocated environment. Shared starting state stays shared, while divergent writes create the new storage demand. That is a different economic proposition from paying for full-capacity copies.

DeltaBox, a 2026 systems research paper identified as arXiv:2605.22781, addresses millisecond-level sandbox checkpoint and rollback for stateful AI agents. Anthropic’s 2026 documentation also treats checkpointing as a first-class session management feature. Fast resumption is becoming normal workflow machinery. With persistent, forkable disks such as those Archil provides, checkpoints can take milliseconds regardless of disk size, allowing context snapshots to support parallel runs without copying every byte. The warm base environment serves the same purpose as a stable prompt prefix, which is to reuse preparation.

Giving agents file access to large datasets without copying them into the sandbox

A dataset copied into a sandbox can become stale as soon as its source changes. Copying also creates a second stored version, so the operator pays for duplication as well as risking outdated results. Mounting the authoritative bucket avoids making a private copy the default working dataset.

Amazon S3 Files, launched April 7, 2026, mounts an S3 bucket into an agent’s local environment using a stage-and-commit model. Changes accumulate on the filesystem side and are pushed back as whole objects, preserving guarantees existing S3 applications depend on. Another option is Archil’s cloud file system, which mounts S3, GCS, R2, and Azure buckets directly and provides an S3-compatible API on every disk. Its active-use billing applies to the filesystem layer, while the underlying bucket’s pricing still governs stored data, requests, and any transfer charges.

The interface has practical value beyond billing. Models are disproportionately trained on bash and command-line tools, so standard file operations avoid bespoke MCP tools that consume context. S3, GCS, Azure Blob, and R2 are strongly consistent per key, but the filesystem layer must still provide durable write behavior. An agent needs fsync semantics to know a write is safe before proceeding.

How do you give each customer an isolated storage workspace at thousands of tenants?

Provisioning peak workspace capacity for thousands of tenants means reserving space for thousands of simultaneous peaks, even though most tenants are idle most of the time. The multiplication is unforgiving. A bill that looks tolerable for one customer can consume the platform’s margin when repeated across every inactive account. Active-data billing charges for storage that is resident rather than adding up the size of all provisioned volumes.

The reliable isolation pattern is a workspace per tenant, with every file operation tied to a specific workspace ID. A shared bucket separated only by application logic risks exposing another tenant’s files through one bug. Tenant-scoped secrets must be injected at the storage or network boundary. When tenants share a credential, a leak in one session puts the whole platform at risk.

Amazon Bedrock AgentCore Runtime provides a concrete execution model through session-isolated microVMs, each with an isolated filesystem. Its compute is ephemeral by default, so persistence requires separately configured session storage. That distinction is essential. Session isolation prevents sessions from sharing an execution filesystem, while tenant workspaces preserve the customer’s state across the required lifecycle.

Shared infrastructure also needs per-tenant token-bucket rate limiting so one customer’s parallel attempts do not degrade storage performance for everyone else. As with shared tables, separate schemas, and separate databases, stronger filesystem separation brings provisioning and operational tradeoffs. Billing must not turn those boundaries into thousands of idle-capacity charges. Together, these constraints define what the storage model has to deliver.

What the right storage billing model for agent workspaces looks like when you add the requirements up

Active-data billing fits these workloads because it does not penalize the retained workspace capacity that reusable context, parallel exploration, and tenant isolation require. The billing model must charge differently for durable state and data that is actively resident. Otherwise, “persistent” becomes another name for “always billed at peak.” Underlying object storage charges remain part of the calculation, but they need not be joined by a fully provisioned volume for every idle agent.

The execution environment needs a durable filesystem that survives stops, with idle compute able to pause and resume quickly. Branching needs copy-on-write or write-ahead-log-based snapshots, with charges for the data written. To use datasets, agents need mounts over buckets they already have, durable writes, and transfer pricing that keeps the cost of repeated reads down. Tenant isolation still requires scoped credentials, controlled egress, and separate workspace boundaries regardless of the billing label.

Cheap storage cannot repair weak isolation, and persistent files cannot guarantee a provider cache hit. But when the prefix remains reusable and the provider serves it from cache, the benefits compound. The agent pays for active use at the storage layer and earns the cache-read rate at the inference layer, so preserving useful context becomes an advantage on both bills.