How a Write-Ahead Log Keeps a Filesystem Consistent

A filesystem that crashes mid-write has no way to tell a good state from a bad one on its own. Directory entries can end up pointing at blocks that were already freed, data can go orphaned with nothing referencing it, and inode tables and bitmaps can land half-updated. None of this announces itself. The filesystem has no internal flag that says "this metadata is broken," so it needs an outside process to check its work. For decades that process was fsck, a full scan of the disk that could take hours on a large drive and locked everyone out of the filesystem while it ran. The idea that replaced it is simple to state and hard to get wrong in practice: write down what you're about to do, durably and in order, before you do it. A crash no longer means digging through the wreckage to guess what happened; it means replaying the log and knowing what happened.

What the journal structure looks like on disk

The journal is a bounded container built from three pieces, repeated for every transaction that passes through it. The first piece is the Transaction Begin Block, or TxB, which carries a transaction ID (TID) unique to that operation. The second piece is the payload itself: copies of the inode, bitmap, and data blocks as they will look once the change is applied. The journal doesn't store a diff describing what changed. It stores the finished product, ready to be dropped into place. The third piece is the Transaction End Block, or TxE, which closes out the transaction under that same TID. Its presence on disk is the signal that the transaction committed. If it's missing or corrupted, the whole transaction gets thrown out during replay.

The journal doesn't grow forever. It runs as a circular buffer: once a transaction has been checkpointed into the real filesystem, it gets marked free, and the superblock's pointer to the oldest live transaction moves forward to reclaim that space. Each entry also carries a monotonically increasing offset, called an LSN, so a recovery process can tell at a glance which entries have already been applied and which haven't. That LSN mechanism reappears later, both in how checkpoints limit recovery time and in how object storage systems try to recreate journaling without a traditional disk underneath them.

The atomic commit sequence: the three steps that enforce the guarantee

The entire WAL guarantee comes down to three steps happening in a fixed order. Get the order wrong, and the guarantee stops meaning anything.

Phase 1 is the journal write. The TxB, the inode, the bitmap, and the data blocks all get written into the journal first. Nothing on the real filesystem has been touched yet. This is staging rather than committing.

Phase 2 is the journal commit, and it's a single, specific write: the TxE block. It only gets written once every payload block from Phase 1 is confirmed durable on disk. If TxB, the payload, and TxE all went down at once, a crash partway through could leave TxE sitting on disk while the payload behind it is incomplete, which would make a broken transaction look finished. Splitting the journal write from the journal commit is what prevents that false signal.

Phase 3 is the checkpoint: the inode, bitmap, and data blocks finally get written to their real, final locations on the filesystem. Only now does the actual filesystem change.

Walking through what happens if the system dies at each phase shows why the design works. A crash during Phase 1, before TxE lands, means recovery scans the log, finds no TxE, and throws the transaction out entirely. The filesystem sits exactly where it was before the operation started, which was already a consistent state. A crash after Phase 2 but before the checkpoint finds a complete transaction sitting in the log, TxB and TxE both present, and replays it. That replay works because the journal already holds the finished version of every block, so applying it twice produces the same result as applying it once. A crash during the checkpoint itself, Phase 3, gets handled the same way: redo the already-committed transaction. Some blocks might get written twice, but nothing ends up wrong.

There's a performance footnote buried in this design that's easy to miss. Appending to a log is a sequential operation, while updating scattered filesystem locations directly is a random one, and sequential writes are simply faster. The WAL turns what would have been scattered, random disk activity into one steady stream of appends. That's part of why paying for a double write doesn't sink performance in practice.

The three journaling modes and their trade-offs

Journaling is a dial, and where you set it trades safety for speed in a specific, measurable way.

Full journaling, also called data journaling, writes both the metadata and the actual data into the journal before either reaches its final home. Full journaling is the safest option and the most expensive one, write-wise; it shows up in places where losing or corrupting data isn't an acceptable risk, financial systems being the textbook case.

Ordered mode only journals the metadata, but it forces the actual data to hit disk before the metadata commit happens in the journal. That ordering closes a specific hole: metadata can never end up pointing at new data that was never actually written. This is the default mode for ext4. XFS, by contrast, defaults to journaling metadata only, without that same guarantee that data lands before the metadata commit.

Writeback mode also journals metadata only, but drops the ordering constraint. Data writes and journal commits happen independently, with no guarantee about which lands first.

None of this is academic. Ordered mode's balance of safety and speed is why it's the default across major Linux filesystems, while writeback mode only makes sense where the application sitting on top of the filesystem can tolerate, or at least detect, reading stale data. A fair question is whether journaling only metadata in ordered mode gives up data integrity. It doesn't: because data is guaranteed to hit disk before the metadata commit that describes it, a post-crash replay of that metadata can never point to data that doesn't exist yet.

The double-write cost and the checkpoint trade-off

Nothing about WAL is free. It introduces two costs that compound if left unmanaged: every change gets written twice, once to the log and once to its final location, and the longer the log grows, the longer recovery takes. Checkpointing exists specifically to keep both of those costs in check.

The double write is baked into the design. A write goes to the log, then gets rewritten to its final location. That's the price paid for a guarantee that recovery is always deterministic.

Where this gets expensive is at the layer sitting on top of the WAL. The log's own writes are sequential and efficient, but databases built on top of WAL often add their own in-place writes, and those can multiply total flash writes well past what the application actually asked for. On an SSD, that difference is not cosmetic: every write wears down flash cells, and a system careless about write amplification can burn through a drive's rated endurance far faster than expected.

Recovery cost scales the same way, in the other direction. If the WAL is the only record of what state the system should be in, recovery means reading every log record since the last checkpoint and reapplying them in order, so a longer unreplayed log means a longer wait before the system is usable again. Checkpointing solves this directly: periodically dumping the application's state to stable storage means everything logged before that checkpoint can be thrown away, which caps both the size of the log and how long replay takes. Batching durability flushes helps on the write side too. Flushing every single log write individually is slow, and grouping several entries into one flush cuts that cost without giving up the durability guarantee.

The more interesting work happening right now is in rethinking where writes land in the first place. Out-of-place write strategies, the kind described in LeanStore's redesign away from in-place B-tree updates, cut total write amplification by handing the database control over exactly where writes go, especially when combined with other optimizations spanning the database and the SSD itself.

Crash recovery replay: how the log restores a consistent state step by step

Recovery follows mechanically from how the commit sequence was written.

On restart, the filesystem scans the journal looking for valid, complete transactions, checking for TxE presence and validating CRCs to separate the finished transactions from the partial ones. Anything missing a TxE gets skipped. The filesystem falls back to whatever state existed before that transaction started, and that prior state was consistent by definition, so there's nothing to fix. Transactions that are complete, TxB and TxE both present, but hadn't been checkpointed yet, get replayed through redo logging. Running that replay twice produces the exact same result as running it once, which is what makes it safe to redo a transaction even if part of the checkpoint had already gone through.

The LSN on each entry is what keeps this efficient. Recovery doesn't need to re-read the entire journal from the beginning. It only needs to look at entries after the last confirmed checkpoint LSN, because everything before that point is already known to be safely applied. The superblock's pointer to the oldest unfreed transaction narrows the scan further, limiting it to the live portion of the circular buffer rather than the whole thing.

This replay logic is not unique to one filesystem. Linux ext3, ext4, and Windows NTFS all implement this same recovery model. The mechanism generalizes because the underlying problem, recovering a consistent state after an interruption, generalizes.

Adapting WAL to object storage, where POSIX write semantics do not exist

That generality gets tested hard once the storage layer stops being a traditional disk. Object storage systems like S3 and GCS don't support in-place atomic updates to arbitrary byte ranges. Azure Blob page blobs are the one exception, supporting in-place atomic writes, but only to 512-byte-aligned page ranges. Classical journaling leans on exactly the capability object storage mostly doesn't have: overwrite this one block (the TxE block) to signal a commit. Without that primitive, the commit signal at the center of the whole protocol needs a different implementation.

The guarantee itself still holds up, because it never actually depended on disks specifically. It depended on having a durable, sequential, checkable record of intent before anything destructive happens, and object storage's atomic primitives, limited as they are compared to a POSIX filesystem, can still be arranged to produce that same record. OSWALD, short for Object Storage Write-Ahead Log Device, is one approach built directly around that constraint. The goal carried over unchanged: a durable log the system can trust more than the state of the data itself, and crash recovery becomes log replay, not disk archaeology.