How Read-Ahead and Prefetching Work in Storage Systems

Storage systems have been guessing what you want to read next for decades. Read-ahead and prefetching are the two techniques behind that guess, and both work by moving data into memory before an application asks for it, so a blocking read turns into background work that never stalls the thread. Understand how they detect patterns, size their windows, and interact with cache, and you understand why every fast storage system, including the ones built over object storage, depends on them.

Read-ahead handles predictable sequences and prefetch handles randomness, and confusing the two leads to picking the wrong tool for the access pattern. Read-ahead fires when the system spots a run of consecutive blocks being read and races ahead to grab more before you ask. Prefetch is the cousin that handles randomness: read a small chunk of a file, and the system bets the rest of that block is coming next, so it grabs the whole thing asynchronously. Neither one happens where you'd expect. Host computers mostly don't issue these speculative reads themselves. The work happens inside cache management software or the firmware sitting on a storage controller, quietly, without the host knowing much about it at all. The payoff is the same either way: a synchronous read that would've frozen your application becomes a background fetch that doesn't. That's why streaming media, database table scans, and large sequential file jobs benefit so much. The pattern is predictable, so the guess is usually right. Object storage changes the math on that guess, because the assumption that a guess is usually right no longer holds.

How adaptive window sizing calibrates prefetch depth without wasting I/O

A fixed-size prefetch window is a blunt tool. Too small and you leave latency on the table. Too big and you're burning I/O budget fetching data nobody asked for. So real systems watch what actually gets used and adjust the window size on the fly, a feedback loop rather than a fixed setting.

ZFS is a clean example of this in practice. According to Klara Systems, the prefetch window starts out sized to match the application's first read request. From there, every time the prefetched blocks actually get consumed within a few seconds, the window doubles. That doubling keeps going until it hits 4 MiB. Past that point, growth slows down: instead of doubling, the window grows by just one-eighth of its current size, capping out at 64 MiB.

That two-speed design is the whole point. Aggressive doubling early on lets the system commit hard to a stream that's clearly sequential. But if growth kept doubling forever, one long-running sequential read could balloon the prefetch window into fetching tens of megabytes of data the application might never touch. The one-eighth increment past 4 MiB is the brake pedal. It's not that ZFS stops trusting the stream, it just stops betting the whole farm on it.

Research on GPU file system readahead (arXiv 2109.05366) describes a similar multi-stream readahead model where concurrent read streams are each tracked and calibrated independently. Research on GPU file system readahead (arXiv 2109.05366) describes a multi-stream readahead model where concurrent read streams are each tracked and calibrated independently. That means several concurrent read streams, each from a different thread, can each get calibrated the right way without stepping on each other. None of this needs the application to tune anything. The system watches its own predictions, checks whether they paid off, and adjusts. Self-correcting, not hand-tuned.

Where pattern detection happens, from OS kernel to SSD firmware

Pattern detection doesn't live in one place. It happens at three different layers, and each one catches something the layer above or below it misses.

At the top, inside the Linux kernel, the page cache tracks file access and triggers readahead when it spots sequential reads. One layer down, filesystems like ZFS run their own prefetch subsystem, which has more context than the kernel because it understands file structure and metadata directly, not just raw block addresses.

Then there's the layer almost nobody thinks about: the SSD itself. Flash translation firmware inside the drive does its own pattern detection, entirely hidden from the operating system. A study published in ACM Transactions on Storage measured this and found average read latency improvements ranging from 1.8% to 36.5%, with no meaningful hit to write latency. The host never issues a prefetch command in this case. The drive just watches the I/O it's been handed and acts on its own. That's a strength, since there's zero integration work required, but it's also a real limit: the drive has no idea what the application is actually trying to do. It only sees block requests, not intent.

ZFS's cache layer, the ARC (Adaptive Replacement Cache), adds another wrinkle. Unlike a plain least-recently-used cache, ARC tracks both recently used and frequently used data and shifts the balance between them based on the workload. Prefetched data lands directly in ARC, so the prefetch mechanism and the cache aren't separate systems bolted together, they're deeply coupled. That's fine when there's one layer doing this work. It gets more complicated when a filesystem built on top of object storage adds its own prefetch logic on top of an OS that's already trying to do the same job. More on that shortly.

Why object storage breaks the assumptions underlying traditional prefetch

Traditional prefetch was designed around local disk seek times, low single-digit milliseconds. Object storage blows that assumption apart. A request to a remote bucket runs an order of magnitude slower, and every assumption baked into decades of prefetch design has to be rebuilt around that fact.

According to a JuiceFS engineering blog post, access latency to object storage without readahead typically runs above 10 milliseconds. Turning readahead on drops access latency to around 100 microseconds, two orders of magnitude faster than the 10 milliseconds without it. That's not a tuning win, that's two orders of magnitude. And the gap isn't cosmetic. At 10 ms per access, certain streaming and interactive workloads are simply unusable. At 100 microseconds, they're fine. Prefetch turns a workload from unworkable into workable, well beyond a mere performance polish over object storage.

There's a second assumption object storage breaks, one that's easy to miss. On a spinning disk, "the next block" is a physical location, right next to the one you just read. Objects in a bucket don't work that way. There's no platter, no physical adjacency to lean on. So the storage layer has to invent adjacency logically, usually through some kind of chunking scheme that defines what "nearby" even means. JuiceFS handles this by splitting files into 64 MB chunks, then subdividing those into 4 MB blocks, and those 4 MB block boundaries, not raw byte offsets, become the actual unit that prefetch operates on. None of this makes object storage a lesser option. It makes prefetch load-bearing in a way it never had to be on local disk, because the cost of guessing wrong is so much higher.

How JuiceFS implements readahead, prefetch, and cache over object storage

JuiceFS is a POSIX-compatible filesystem that sits on top of object storage, reachable through a FUSE mount, the HDFS API, an S3 gateway, a Kubernetes CSI driver, or WebDAV. It works across more than 15 object storage backends, covering the major public clouds as well as self-hosted setups. The chunking structure mentioned above produces that behavior: a file breaks into 64 MB chunks, each chunk splits into 4 MB blocks, and each of those blocks is the actual unit fetched from object storage.

According to JuiceFS's engineering blog, every 4 MB block inside the current readahead window gets its own goroutine to fetch it, and the number of these running at once is capped by a buffer-size setting. So the concurrency isn't unlimited, it's a dial someone can turn.

JuiceFS's documentation on AI workload optimization also describes a handful of tunable settings. There's a max-read-ahead ceiling that caps how far the window can grow, an initial-read-ahead setting for the starting window size, and a read-ahead ratio that governs how aggressively the system prefetches during large-file random reads specifically to control read amplification. The honest trade-off is this: prefetch aggressively during a workload with random access, and plenty of that fetched data never gets read. The ratio setting is how an operator decides how much waste they're willing to eat in exchange for lower latency on the reads that do land.

For sequential scans across large files, JuiceFS also ramps concurrency up automatically, aiming to use as much of the available cache and object storage bandwidth as it can. The overall read path, FUSE layer, readahead buffer, cache, then object storage, mirrors the layered pattern-detection story from earlier. Same idea, just stacked on top of a backend that behaves nothing like local disk.

Measured impact: prefetching's effect on query engines and database workloads

A 2025 VLDB workshop paper studying caching in cloud-hosted relational databases found that turning on file prefetching cut query run time by somewhere between 20% and 51%, depending on the query.

That spread, 20% to 51%, tells its own story. Queries dominated by sequential scans capture most of the benefit, because that's exactly the access pattern prefetch is built for. Queries with unpredictable, scattered access patterns get less out of it, for the same reason random reads get less benefit than sequential ones everywhere else in this piece. It's the same distinction from the very first section, appearing again in a database engine instead of a disk controller.

This matters because database performance conversations usually orbit around indexes, query planning, and CPU. Storage prefetch rarely gets invited to that conversation. A technique that swings query time by up to half frequently is the bottleneck, not a rounding error underneath it. If a database workload sees that kind of swing, the I/O demand at a scale orders of magnitude bigger means a stall doesn't just slow a query, it idles a GPU that costs tens of thousands of dollars.

The AI training storage bottleneck and prefetching's role in it

The scale here is hard to overstate. Global AI infrastructure spending crossed $250 billion in 2025, and storage and networking are growing rapidly alongside compute. One estimate puts the AI storage market growing nearly tenfold from 2025 to 2035.

None of that spending fixes idle GPUs, the actual problem at the center of AI training. Enterprise GPU utilization has been reported as low as 5% when the data pipeline can't keep pace with the compute. A GPU cluster is bought to crunch numbers, not to sit around waiting for a data loader to catch up, and yet that's exactly what happens when storage can't feed it fast enough.

Meta's own documented experience, reported by introl.com, put a number on this: 56% of GPU cycles stalled out waiting on training data, because storage capacity couldn't keep up with the pace training demanded. Meta's fix was building a dedicated Data PreProcessing Service to run at scale. Sit with that 56% figure for a second. More than half of the most expensive hardware in the building was doing nothing, and the fix wasn't a faster GPU, it was a storage and prefetch problem.

Training workloads make this hard because they demand two very different I/O patterns at once: billions of small random reads while loading training data, and terabyte-scale sequential writes during checkpointing. A prefetch strategy has to handle both, often at the same time, not one or the other. Production practice includes epoch-pattern-based prefetching and warming the cache during off-peak hours to get ahead of the load, as introl.com describes.

There's also a subtlety specific to training that prefetch has to respect: randomness is a feature. U.S. Patent 12,164,812 describes a prefetch architecture that pulls from a list of batch storage locations and deliberately applies randomization to that list rather than reading it in sorted order. Training data has to stay shuffled, or the model learns the sort order instead of the actual patterns in the data. Prefetch here isn't just about speed, it has to preserve statistical randomness while still guessing right.

Why training and inference need different prefetch architectures

Training and inference want opposite things from storage. One prefetch design can't serve both well.

Training rewards sequential throughput: big block sizes, deep I/O queues, and prefetching that leans into long, predictable sequential reads. Checkpoint writes and data loading both benefit from the kind of adaptive window growth described earlier in this piece. Inference wants the opposite: small blocks, shallow queues, and unpredictable access patterns where latency affects performance more than raw throughput. Pointing an aggressive sequential prefetcher at an inference workload hurts it, pulling in data nobody needs and driving up read amplification exactly when low latency matters most.

This is a genuine fork in design, not a dial you turn between two settings on the same system. It's a genuine fork in design. A storage system tuned for training throughput delivers poor inference latency, and a system tuned for inference latency can't feed a training cluster fast enough to keep GPUs busy. Sequential prefetching belongs to training. Parallel, random-read optimization belongs to inference. Trying to make one architecture do both is how you end up mediocre at each.

Recent research gets specific about the inference side. Recent research describes overlapping data movement with GPU computation directly: while the GPU is executing one layer of a model, the system is already prefetching data for the next layer, hiding the transfer time behind work that's already happening. Direct memory access keeps the host CPU out of that critical path entirely. Separately, a system called DirectKV, presented at OSDI '26, gets rid of the GPU staging buffer altogether, letting GPU kernels reach directly into CPU-resident key-value cache memory instead of copying it back and forth first. Both are inference-specific fixes, and neither would make sense applied to a training pipeline.

Anyone building this infrastructure should figure out whether training or inference is the dominant workload before picking a prefetch strategy, not after the hardware's already racked.

GPUDirect Storage and prefetch bypassing the CPU entirely

GPUDirect Storage (GDS) takes prefetch a step further by cutting the CPU out of the path completely. Data moves directly from NVMe storage into GPU memory, with no CPU staging buffer and no serialization step in between. It's a zero-copy path, storage straight to GPU.

The performance gain traces directly back to prefetching. NVIDIA's own reported 15% training throughput improvement from GDS comes specifically from eliminating CPU I/O stalls inside the DataLoader's prefetch pipeline, the same handoff point that chokes so many training jobs. Scality has reported that for I/O-bound training specifically, GDS can push throughput up 20% to 50% just by removing the CPU and memory as bottlenecks in that path.

The hardware backing this has moved fast. GDS 1.7 and later, shipping with CUDA 12.2 and up per NVIDIA's documentation, includes native support for H100 and H200 GPUs. A single PCIe Gen5 NVMe drive can hit 14 GB/s, and stacking several per server gets north of 400 GB/s. NVIDIA's Magnum IO stack, tuned for Blackwell, has shown early benchmark numbers around 250 GB/s sustained. AWS, Azure, and another major cloud provider all now offer GDS-enabled instances.

Scaling this out across a cluster makes the numbers bigger still. Spheron has reported that a 32-GPU cluster running InfiniBand or RoCE networking with a GDS-compatible backend can pull over 100 GB/s of aggregate read bandwidth just for dataset prefetching. At the far end of that curve, introl.com reports DDN's EXAScaler system delivering 4 TB/s to NVIDIA's Eos supercomputer. And the idea of accelerator-driven readahead isn't purely a hardware story either: Technion researchers building a GPU I/O readahead prefetcher into GPUfs (arXiv 2109.05366) measured more than double the bandwidth in microbenchmarks, and across 14 applications pulled from the RODINIA, PARBOIL, and POLYBENCH benchmark suites, execution time improved by as much as 50%.

Prefetch started as a trick to avoid making an application wait on a spinning disk. It's ended up as the thing deciding whether a GPU cluster costing millions of dollars actually spends its time computing, or just sits there, waiting.