Journaling File Systems
AdvancedJournaling file systems use a write-ahead log to record intended changes before applying them, ensuring filesystem consistency can be recovered after a crash.
Overview
Without journaling, a crash mid-write can leave a filesystem in an inconsistent state (e.g., a file's directory entry exists but its inode is not initialised). Traditional recovery required a full fsck scan — slow on large disks. Journaling solves this with a write-ahead log (WAL): changes are first written atomically to the journal, then applied to the filesystem. On recovery, the kernel replays committed journal entries and discards incomplete ones. Three modes exist: metadata-only journaling (ext3 default — fast, journal only tracks inode/directory changes), ordered mode (metadata journaled, data written first), and full journaling (both data and metadata — slowest, safest). ext4, XFS, APFS, NTFS, and HFS+ all use journaling.
How the Journal Works
A write operation generates a journal transaction: (1) write a begin marker, (2) write the changed blocks to the journal, (3) write a commit marker. Only after the commit marker is on disk does the OS apply changes to their real locations and mark the journal entry as free. On crash, the OS scans the journal for committed-but-not-applied transactions and replays them.
// Journal transaction lifecycle (pseudocode)
class JournalTransaction {
int transactionId;
List<JournalBlock> blocks; // changed data/metadata blocks
// Phase 1: Write changes to journal (sequential I/O — fast)
void writeToJournal(Journal journal) {
journal.write(new BeginMarker(transactionId));
for (JournalBlock b : blocks) {
journal.write(b); // write block to journal region
}
journal.write(new CommitMarker(transactionId));
journal.flush(); // fsync: ensure commit hits disk before proceeding
}
// Phase 2: Checkpoint — write to actual filesystem locations
void checkpoint(Filesystem fs) {
for (JournalBlock b : blocks) {
fs.writeBlock(b.targetLocation, b.data);
}
fs.flush();
journal.freeTransaction(transactionId); // journal space reclaimed
}
}
// Recovery on mount after crash:
void recover(Journal journal, Filesystem fs) {
for (JournalTransaction tx : journal.committedTransactions()) {
// Safe to replay: commit marker exists, so data is complete
tx.checkpoint(fs);
System.out.println("Replayed transaction: " + tx.transactionId);
}
for (JournalTransaction tx : journal.incompleteTransactions()) {
// No commit marker → crash during write → discard
journal.discard(tx);
System.out.println("Discarded incomplete transaction: " + tx.transactionId);
}
}Java fsync and Relation to Database WAL
FileChannel.force(true) maps to the fsync system call — it flushes both data and metadata from OS buffers to physical disk. Databases use the same journaling concept as their Write-Ahead Log (WAL): PostgreSQL, MySQL InnoDB, and SQLite all write WAL records before modifying data pages. Understanding this is essential for writing durable Java file I/O.
import java.nio.file.*;
import java.nio.channels.*;
import java.nio.ByteBuffer;
// Durable write: ensure data survives a crash before confirming to caller
Path logFile = Path.of("/tmp/wal.log");
try (FileChannel channel = FileChannel.open(logFile,
StandardOpenOption.CREATE,
StandardOpenOption.WRITE,
StandardOpenOption.APPEND)) {
String entry = "TXN-42: UPDATE account SET balance=500 WHERE id=1
";
ByteBuffer buf = ByteBuffer.wrap(entry.getBytes());
channel.write(buf);
// force(true): flush data AND metadata (file size, mtime) to disk
// force(false): flush data only (faster, sufficient for append-only logs)
channel.force(true); // equivalent to fsync() — BLOCKS until disk confirms
System.out.println("WAL entry durably written.");
// Only AFTER this point is it safe to acknowledge the transaction
}
// FileOutputStream equivalent (less precise control):
try (var fos = new java.io.FileOutputStream(logFile.toFile(), true)) {
fos.write("entry
".getBytes());
fos.getFD().sync(); // maps to fsync() — flushes OS buffers to disk
}
// Java NIO2 atomic move (used after writing temp file — crash-safe rename):
Path tempFile = Path.of("/tmp/config.tmp");
Files.writeString(tempFile, "new config");
Files.move(tempFile, Path.of("/tmp/config.cfg"),
StandardCopyOption.ATOMIC_MOVE, // OS-level atomic rename
StandardCopyOption.REPLACE_EXISTING);Key Points to Remember
- 1Journaling ensures filesystem consistency after a crash by replaying committed journal entries.
- 2Metadata-only journaling is the default in ext3/ext4 — faster, but a crash can corrupt file data (not just metadata).
- 3Full journaling protects both data and metadata but roughly halves write throughput due to double-writing.
- 4FileChannel.force(true) in Java maps to fsync() — mandatory for durable writes in critical applications.
- 5Database WAL (Write-Ahead Log in PostgreSQL, InnoDB redo log) is the same concept applied at the database layer.
- 6Files.move() with ATOMIC_MOVE uses the OS rename() syscall — safe replacement pattern for updating config files.
Interview Questions
Sign in to ask AriaWhat problem does journaling solve, and how does it work?
What is the difference between metadata-only journaling and full journaling in ext4?
How does FileChannel.force(true) relate to the OS fsync system call, and why is it important?
How is a database Write-Ahead Log similar to filesystem journaling?
Ask Aria about Journaling File Systems
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.