How a Write-Ahead Log Keeps a Filesystem Consistent
A file write that looks like one action is actually three separate writes to three separate structures on disk, and if the power dies between any two of them, the filesystem ends up in a state that never should've existed. That's the whole problem a write-ahead log solves. Nothing more, nothing less.
Picture appending a few lines to a log file. The mental model most people carry around is: you hit save, data goes to disk, done. One motion, one outcome. That model is wrong, and it's wrong in a way that matters. Appending to a file actually triggers at minimum three distinct writes: the inode gets updated with the new file size and block pointers, the allocation bitmap gets flipped to mark the new block as used, and the data itself gets written to that block. Three writes, three locations on disk, zero built-in guarantee they happen together.
So what happens if the machine loses power after step one but before step three? The inode now claims the file is bigger than it actually is. It points to a block that's marked allocated but contains garbage, or worse, stale data from whatever used to live there. That's not a crash with a clean, obvious failure. That's corruption, quiet and structural, and the filesystem has no idea anything went wrong until something tries to read that block.
This is a flaw baked into how storage stacks are built generally, not specific to one filesystem or one vendor's engineering team cutting corners. It's baked into how storage stacks are built. Every layer assumes the layer below it is atomic, and no layer actually delivers that atomicity for free. The disk doesn't guarantee it. The block layer doesn't guarantee it. Somebody has to build the guarantee on purpose, and that's the job description for a write-ahead log.
Faster hardware doesn't fix this either. A big enough storage drive or a fast enough bus might seem like it would close the gap, but speed was never the issue. It's about the fact that a single logical operation (append this file) requires multiple physical writes, and physical writes can always be interrupted between one and the next. Speed shrinks the window. It doesn't close it.
What a write-ahead log does, the rule and nothing more
The rule is short enough to fit on a sticky note: changes to data files get written only after a record describing those changes has been logged, and that log record has to be flushed to permanent storage first. That's the entire mechanism, short enough to fit on a sticky note, because the whole discipline comes down to sequencing, not cleverness. That's the entire mechanism. The whole discipline comes down to sequencing, not cleverness.
The log itself is boring by design. It's append-only, it's sequential, and it lives on disk as a running record, never edited in place. "Write-ahead" describes a causal order: the log entry always comes before the data write. Always. Not usually, not when convenient.
Here's what that ordering buys. If the system crashes after the log write but before the data write, the log record already made it to durable storage. On restart, the system can replay that record and finish the job the crash interrupted. The data write was lost, but the intent to make it wasn't, and intent that survives a crash is recoverable.
Worth contrasting against the older approach, the rollback journal, which predates WAL mode in SQLite. Rollback journaling copies the original content into a side file and then edits the main file directly. WAL flips that completely. The original content stays put in the main file, untouched, and every change gets appended to the separate WAL file instead. A commit, in WAL mode, is just a special record appended to that log. The main data file doesn't get touched until a checkpoint happens later.
None of this stops crashes from happening. Power still goes out, processes still die mid-write. What the WAL buys is determinism: after a crash, the system always knows exactly what state it's in and exactly what to do about it.
What happens inside a crash and a recovery, the sequence step by step
Two scenarios cover basically everything. Scenario one: the crash happens before the log write finishes. On restart, there's no complete log record for that operation, so the system treats it as though it never started. Nothing to replay, the data file was never touched, consistency intact. Clean.
Scenario two: the crash happens after the log write completes but before the data file gets updated. This time the log record is sitting there, complete and durable. Recovery finds it and replays it forward, a process PostgreSQL's documentation calls roll-forward recovery or REDO. The data file gets brought up to the state the original operation intended. The crash cost some time, not correctness.
Wikipedia's entry on write-ahead logging states that logs typically carry both redo and undo information. Redo finishes what should've finished. Undo handles the opposite case, a transaction that was partway applied but should never have been committed in the first place, and needs to be unwound rather than completed.
The log functions as the single source of truth here. Recovery isn't guessing. It compares what the system was supposed to be doing against what actually landed on disk, then picks one of three moves: undo it, finish it, or leave it alone because it was already complete. And because the log is sequential, recovery reads it in order with no random seeking around the disk hunting for inconsistencies. That's a big part of why recovery is fast.
Compare that to the older approach of checking filesystem consistency, the pre-journaling method of scanning the entire disk looking for anything that doesn't add up. Journaling largely eliminates that scan, because every action already got logged before it touched disk, as GeeksforGeeks' explanation of journaling states. No need to inspect the whole disk when the log already tells you exactly what happened and where it stopped.
Individual journal entries typically carry ordering information, a transaction ID, an operation type, and the actual payload. A recovery processor just tracks the last offset it handled, and it knows immediately which entries downstream are still unprocessed. Journal structures also typically include a transaction begin block with its own ID, as GeeksforGeeks explains, giving recovery a clean marker to start from rather than guessing where a transaction began.
How checkpoints bound the log and control recovery cost
Logs can't grow forever. Eventually, something has to stop and say: everything up to this point is safely written to the real data files, so anything before this line can be thrown away. That something is a checkpoint. A checkpoint marks the point where application state has been completely written to stable storage, meaning recovery never has to replay from the very beginning of time.
What a checkpoint actually does is transfer log-recorded changes into the main data files, and then it frees up the log space before that point for reuse. Journals often behave like circular buffers, where newer transactions eventually overwrite older ones once the journal fills up. Overwriting old journal entries would be reckless without a checkpoint guaranteeing those entries already made it into permanent storage. The checkpoint is what makes the circular part safe instead of catastrophic.
SQLite runs this automatically. Once the WAL file crosses a size threshold, a checkpoint fires on its own once the WAL file crosses a size threshold. Applications can also trigger checkpoints manually, usually timed for idle moments when there's no active write traffic to compete with.
Without checkpoints, recovery cost balloons. In the operation-replay log pattern, where the log is the only persistent record and actual state lives only in memory, every single log entry since the beginning has to replay after a crash. Checkpoints exist specifically to put a ceiling on that cost, as the Rust WAL paper explains.
There's a real tradeoff here, and it's not free to ignore. Checkpoint too often, and normal operation eats extra I/O it didn't need. Checkpoint too rarely, and the log balloons, dragging out recovery time whenever a crash actually happens. Frequency is a dial you turn based on workload, not a correctness guarantee. Correctness was already handled by the WAL rule itself, back before checkpointing ever entered the conversation.
How filesystem journaling applies the WAL idea, ext4, Btrfs, and what each protects
Most modern filesystems run some version of a WAL for at least their metadata, and that's what journaling actually is. Two filesystems make the contrast especially clear: ext4 and Btrfs, and they solve the same problem from opposite directions.
ext4 uses a kernel journaling subsystem called JBD2. It logs the metadata changes it intends to make into a journal area before those changes ever touch the main filesystem. After a crash, the journal replays and the filesystem snaps back to a consistent state, quickly, without a full disk scan. Journaling protects filesystem consistency, not your data. Depending on the journaling mode in use, data that was mid-flight during the crash can still be lost even though the filesystem itself comes back clean. ext4 has been running in production long enough that its failure patterns and recovery behavior are well understood, which is a big reason it's still a common default across Linux installs.
Btrfs takes a completely different route to the same destination: copy-on-write. Instead of overwriting data in place, Btrfs always writes to a new location, which makes each write atomic by construction. There's no journal needed for basic crash consistency, because the copy-on-write structure carries its own atomicity guarantee built in. Btrfs still keeps transaction groups and a tree-log for replaying incomplete writes or interrupted tree updates, and it layers on checksumming, which gives it stronger integrity guarantees than journaling alone can offer.
Journaling and copy-on-write are two different mechanisms aimed at the exact same goal, making a multi-step write behave like one atomic action. They differ in where the write goes, not in what they promise.
PostgreSQL's own documentation makes this point too. Since WAL already restores database file contents after a crash, journaled filesystems aren't actually necessary for reliably storing PostgreSQL's data files or WAL files. Journaling overhead can even hurt performance in that context, and it's often disabled at mount time for exactly that reason.
Keep the limitation in focus: journaling buys consistency. It does not buy durability of whatever data was in flight the instant the crash hit. Conflating the two is the single most common misunderstanding people carry into this topic.
A real WAL failure: what the SQLite WAL-reset bug reveals about correctness assumptions
On March 3, 2026, an SQLite developer found and fixed something called the WAL-reset bug, an issue present in every SQLite version from 3.7.0, released July 21, 2010, all the way through 3.51.2, released January 9, 2026. SQLite's own documentation states that the fix landed in version 3.51.3. That's roughly sixteen years the bug sat there, undetected.
The trigger wasn't a crash. It was concurrency: two or more connections to the same database file, running in separate threads or separate processes, attempting to write or checkpoint at the exact same instant. Under that narrow condition, the bug could produce database corruption. Rare, but real.
What this reveals is that WAL's correctness isn't unconditional, it's conditional on the concurrency assumptions underneath it holding up. WAL's correctness depends on the concurrency assumptions underneath it holding up, and when they don't, the guarantee fails. The core log mechanism, the append-then-replay logic covered above, was never broken. The failure lived in the coordination layer between concurrent writers and the checkpoint process. A write-ahead log is a protocol, and every protocol has edge conditions nobody thought to test until sixteen years go by and somebody finally does. It's a protocol, and every protocol has edge conditions nobody thought to test until sixteen years go by and somebody finally does.
The Rust WAL paper states directly that a WAL without reliability is useless, because its entire value proposition rests on the ability to recover data after a failure. A logging mechanism that can itself introduce corruption under the wrong concurrency pattern undermines the reason it exists in the first place.
None of this is cause for alarm about SQLite specifically. If anything, it's the opposite. The bug got found, documented publicly with exact version numbers, and fixed within weeks of discovery. That's what a healthy engineering process around a WAL implementation is supposed to look like.
Why object storage breaks the standard WAL assumptions, and how new primitives are closing the gap
The WAL rule demands a flush to permanent storage before the corresponding data write happens. Object storage spent years unable to reliably promise that, because of eventual consistency: a read right after a write might come back stale, showing you the old version like nothing happened at all.
A latency problem underlies the consistency one. Object store writes typically take tens of milliseconds, which is expensive for a workload like WAL that often consists of many small, frequent writes, the underlying research here shows. Remote block storage options like EBS bring that latency down some, but at added cost and with their own latency floor to contend with.
Consistency has actually moved a lot in recent years, though. On December 1, 2020, S3 shipped strong read-after-write consistency across all operations, at no extra charge, and S3-compatible stores followed that lead. More recently, conditional writes appeared: a PUT operation that only succeeds if a given key doesn't already exist, or only succeeds if it matches an expected version. That's the first honest compare-and-swap primitive object storage has offered, and compare-and-swap is exactly the kind of coordination tool a WAL needs to work safely with multiple writers.
A few concrete systems show what building on that new ground actually looks like. Chroma's wal3 uses Amazon's conditional writes feature (introduced August 2024, with ETag-based writes expanded that November) to let writers say, explicitly, "don't overwrite this file" or "only overwrite it if it matches this exact digest." That lets applications coordinate directly against object storage without bolting on a separate coordination service. Chroma also maintains a running cryptographic checksum over the log content, so the integrity of the whole log can be validated continuously rather than only at read time.
OSWALD takes a stricter approach: a WAL built exclusively out of object storage primitives, requiring nothing beyond read-after-write consistency and compare-and-swap. It runs across AWS S3, Google Cloud Storage, and Azure Blob Storage, supports checkpointing and garbage collection, and has been formally specified and verified using the P programming language, the research behind this piece shows.
BtrLog, accepted to VLDB 2026 in Boston, splits the difference architecturally. It stages writes through a layer of log nodes backed by SSDs for fast, low-latency appends, then asynchronously flushes those writes to object storage in large batched segments. Log records get replicated across a quorum of those SSD-backed nodes in a single network round trip. In evaluation, it aimed to reduce the latency and throughput costs that make object storage challenging for WAL workloads.
None of these three throw out the WAL rule. Log before data write, still the law. The durability mechanism underneath changes, reshaped to fit object storage's particular mix of latency and consistency behavior instead of pretending object storage is just a slow disk. The direction the field is heading looks like a WAL that behaves identically whether it's sitting on S3, GCS, R2, or Azure Blob, with no migration step, no ETL pipeline, and no duplicate copies quietly accumulating cost. Designs that keep compute physically close to storage and treat the bucket itself as the actual source of truth, rather than a backup of one, line up naturally with where WAL-on-object-storage work is heading.
WAL checkpointing at AI training scale, where the mechanism meets GPU economics
AI model checkpoints do the same job as WAL checkpoints. They're durable snapshots of training state that let a run resume after something breaks, instead of starting over from scratch, the research behind this piece shows.
The failure modes even rhyme. An interrupted checkpoint write, or a mismatch in its metadata, can leave the checkpoint unreadable, which is functionally the AI equivalent of a half-written journal entry that recovery can't make sense of. And if reading a checkpoint back off storage isn't parallelized properly, that read delays every GPU waiting downstream, which plays the same role as slow log replay after a crash in a database. Same shape of problem, completely different price tag attached to the delay.
That price tag is not small. A MinIO blog cited in the research behind this piece states that global AI infrastructure spending crossed $250 billion in 2025, and that storage spend has grown substantially alongside overall AI infrastructure investment. More than half of organizations report that data and storage bottlenecks are actively limiting AI performance and scale, and 57% of enterprises say their data flat out isn't ready for AI workloads yet, the same source states.
GPU starvation is the symptom that raises the bill. Slow storage leaves GPUs sitting idle, waiting on data loads or waiting on checkpoints to write or read, industry observers note. It's the exact same pattern as a slow recovery bottleneck on a database, except here the idle cost compounds hour by hour on some of the most expensive compute available anywhere.
There's a specific mechanical bottleneck: the bounce buffer forces the CPU to copy data through an intermediate staging area instead of transferring it directly. For a large checkpoint, the CPU handles the entire round trip, pulling data off the GPU and writing it out to NVMe, and that work competes directly for memory bandwidth against DataLoader workers, NCCL communication, and the OS scheduler all trying to run at the same moment, as has been noted in discussions of AI infrastructure,. Everybody's fighting over the same narrow pipe.
The architectural response mirrors what filesystems already figured out decades earlier. Purpose-built AI storage setups lean on parallel filesystems paired with NVMe-oF and GPUDirect Storage to keep GPUs fed continuously instead of starved. Architectural approaches that reduce data-movement overhead between storage and compute aim to reduce the latency that staging and pipeline steps otherwise introduce, without touching a line of model code.
Which closes the loop the whole piece has been tracing. The exact principle that kept a 1990s filesystem from corrupting itself after somebody tripped over the power cord (sequential, durable writes, logged before the state they describe gets applied) is the same principle keeping a training run with billions of parameters recoverable after a node drops out mid-epoch. Different decade, different price tag, same rule.
