File System Reliability & Virtual File System

When a file is created or grows, the file system must decide which free disk block or blocks to allocate.

This is the dual problem of file layout:

• file layout asks: given a file and an offset, where is the data on disk?
• free-space management asks: given a need for space, which unused blocks should be assigned?

The answer depends on the file organization scheme.

If the file system uses extents, it prefers contiguous runs of blocks, because a large contiguous region can be represented compactly and accessed efficiently.

If the file system uses indexed blocks, then any free block may be acceptable, because the inode or index structure can point to arbitrary blocks.

The two main goals are in tension:

• allocation should be fast;
• allocation should avoid excessive fragmentation.

A simple approach is a bitmap.

Each disk block corresponds to one bit:

• 1 means the block is free;
• 0 means the block is used.

Allocation:

alloc():
    find a set bit
    clear it
    return the corresponding block number

Freeing:

free(n):
    set bit n

Example size calculation:

\[ \frac{4\ \text{TB}}{4\ \text{KB}} = \frac{2^{42}}{2^{12}} = 2^{30}\ \text{blocks} \]

One bit per block gives:

\[ 2^{30}\ \text{bits} = 2^{27}\ \text{bytes} = 128\ \text{MB}. \]

So even a simple bitmap can be large.

The problem is not only storage overhead. A 128 MB bitmap does not fit conveniently in one page and may not fit in CPU caches. If allocation requires scanning the bitmap linearly, allocation can become slow, especially when free bits are far away.

Real systems add structure on top of the raw bitmap.

At a lower level, file systems can identify files by inode numbers.

But users do not want to say:

open inode 742

They want to say:

/home/laurent/pintos/readme

A directory provides the mapping from human-readable names to inode numbers.

On Linux, this mapping is visible with:

ls -i

This prints directory entries together with their inode numbers.

Accessing a named byte range in a file involves two conceptually separate translations.

First:

\[ \text{name/path} \rightarrow \text{inode number}. \]

Second:

\[ (\text{inode number}, \text{file offset}) \rightarrow \text{storage block}. \]

The directory resolves the name.

The inode and its block index resolve the offset.

So the file system has at least three major data structures:

• inode: owns file metadata and points to data blocks;
• free map: decides which blocks are free or allocated;
• directory: maps names to inode numbers.

In simpler file systems, such as FAT, a file has exactly one name.

In this design:

• metadata can live directly in the directory entry;
• there is no need for a separate inode abstraction;
• sharing the same file under multiple names is not naturally supported.

This was historically associated with DOS / early Windows-style file systems.

Unix takes a different view: a file may have multiple names.

This requires an extra level of indirection:

\[ \text{name} \rightarrow \text{inode number} \rightarrow \text{file data}. \]

A directory entry contains:

• a name;
• an inode number.

Multiple directory entries may point to the same inode.

These multiple names are called hard links.

All hard links to the same inode are equivalent. There is no special “original” name.

Because several names may point to one inode, the inode stores a reference count, also called a link count.

When a new hard link is created, the count increases.

When a name is removed, the count decreases.

The POSIX operation is called unlink rather than delete because removing one name does not necessarily delete the file.

The file is actually deleted only when:

1. no directory entry links to the inode anymore;
2. no process still has the file open.

The lecture focused mainly on the first condition, the link count.

A directory is itself a special kind of file.

It has:

• an inode;
• data blocks;
• metadata.

The difference is that the file system interprets its contents as directory entries.

Users are normally not allowed to write arbitrary bytes directly into a directory file, because doing so could corrupt the directory structure.

Every Unix directory contains two special entries:

• . points to the directory itself;
• .. points to the parent directory.

These entries are actually represented in the directory structure, not just imaginary syntax.

They are conventionally maintained by the file system.

A Unix-like directory file is a sequence of records.

A simplified directory entry format is:

#define MAXNAMELEN 255

struct dir_entry {
    uint32_t d_ino;      /* inode number */
    uint16_t d_reclen;   /* length of this record */
    uint8_t  d_type;     /* file type, redundant with inode */
    uint8_t  d_namelen;  /* length of name */
    char     d_name[MAXNAMELEN + 1];
};

Important fields:

• d_ino gives the inode number;
• d_reclen gives the length of the whole record;
• d_type stores the file type;
• d_namelen stores the length of the name;
• d_name stores the file name.

The record length makes the directory behave like a linked list: to find the next entry, add d_reclen to the current entry’s address.

The file type is already stored in the inode.

However, storing it in the directory entry is an optimization.

For example, ls -l or a graphical file browser often wants to know whether entries are files or directories.

Without d_type, the system would need to load every inode just to display the directory.

With d_type, it can often list entries without loading every inode.

File names have variable length.

Using fixed-size 256-byte entries for every name would waste space.

So directory records are sized according to the actual name length plus the fixed metadata fields.

This makes storage compact, but creates internal holes when entries are deleted.

To look up a name in a simple directory:

1. start at the beginning of the directory file;
2. read one directory entry;
3. compare the stored name with the target name;
4. if it matches, load the inode;
5. otherwise, advance by d_reclen;
6. repeat until found or end of directory.

This is a linear scan.

The VFS layer can speed up repeated lookups using the dentry cache, which caches recently resolved directory entries.

Creating a file requires inserting a new directory entry.

The file system must:

1. allocate or initialize an inode;
2. find space in the directory file for the new name;
3. write the directory entry;
4. increment the inode’s reference count.

Because names must be unique inside a directory, the file system must scan the directory anyway to check that the name does not already exist.

During that scan, it can also look for a hole large enough for the new entry.

A hole can be detected using:

If a suitable hole exists, the new entry can reuse it.

If no hole exists, the directory entry is appended at the end.

If the end of the directory block is reached, the directory may need to grow by allocating another data block.

Deleting a name is also a directory operation.

The system:

1. finds the directory entry by name;
2. removes the entry;
3. updates the record lengths to cover the removed space;
4. decrements the inode’s link count;
5. if the link count becomes zero, frees the inode and its data blocks.

There is no general reverse lookup from inode number to all names pointing to it.

Therefore deletion normally starts from the directory name, not from the inode.

If the deleted entry is at the end, the directory can potentially be truncated.

If it is in the middle, the previous entry’s d_reclen is extended so that a linear scan skips over the removed entry.

A directory with thousands or millions of entries makes linear scanning expensive.

Lookup, create, delete, and rename may all become slow.

Caching helps repeated lookups, but it does not fully solve creation or collision detection for new entries.

A common optimization is to organize directory entries using a search tree.

Instead of comparing variable-length strings at every step, the file system hashes the name and uses the hash as a fixed-length key.

For example:

\[ \text{hash}(\texttt{foo.txt}) = 0x30. \]

Then lookup uses integer comparisons in a tree, conceptually similar to a B+ tree.

The tree maps from hashed names to directory entries.

Collisions are possible, because different names may have the same hash. This is handled by chaining or by checking the actual name at the leaf.

Examples:

• ext3/ext4 use HTree directory indexing;
• Btrfs also uses tree-based structures.

A subtle point is compatibility.

Old file-system drivers may only understand the traditional linear directory layout.

Therefore, in ext-style designs, the tree index is stored inside the same directory file and arranged so that old scanners can ignore it.

The directory file contains both:

• regular directory entry blocks;
• tree index blocks.

The tree pointers are offsets inside the directory file.

A modern driver can use the tree.

An old driver can still scan the directory linearly and skip the hidden tree-related regions because they are covered by record lengths.

This is a compatibility trick: the on-disk format is extended without making older readers completely unable to read the directory.

A path such as:

/home/laurent/pintos/readme

is resolved one component at a time.

The algorithm is:

1. start at the root directory;
2. look up home in the root directory;
3. load the inode for home;
4. check that it is a directory;
5. look up laurent inside home;
6. load the inode;
7. continue with pintos;
8. continue with readme;
9. return the final inode.

The root directory has a known inode number. In ext-style file systems, the root inode is commonly inode 2.

A simplified recursive lookup is:

lookup(path) {
    return lookup_dir(root_inode, path);
}

lookup_dir(inode, path) {
    name, rest = split(path, '/');

    next_inode_no = scan_dir(inode, name);
    next_inode = load_inode(next_inode_no);

    if (empty(rest))
        return next_inode;

    if (next_inode->type != DIR) {
        errno = ENOTDIR;
        return NULL;
    }

    return lookup_dir(next_inode, rest);
}

Each directory has its own directory contents and, if optimized, its own tree index.

The tree index is not a global index over the whole file system. It only indexes entries inside one directory.

The data structures could represent a hard link to a directory, but the semantics become problematic.

A hard link points to an inode number.

Inode numbers are meaningful only inside one file system.

Therefore hard links cannot cross file-system boundaries.

A symbolic link, or soft link, is a special file whose content is a path string.

It does not point directly to an inode.

Instead, it stores something like:

/data/user/toni

When the kernel encounters the symbolic link during path lookup, it reads this path and continues lookup from there.

Hard link:

• points directly to an inode;
• target must exist;
• increments the inode link count;
• lightweight: just another directory entry;
• cannot point to directories in ordinary user use;
• cannot cross file systems.

Symbolic link:

• stores a destination path string;
• has its own inode;
• may require data blocks for the path;
• target may not exist, giving a dangling link;
• can point to directories;
• can cross file systems;
• can create loops.

When lookup reaches a symbolic link before the end of the path, the kernel reads the link target and restarts lookup.

Example:

/home/toni/file.txt

Suppose:

/home/toni -> /data/user/toni

Then the lookup becomes:

/data/user/toni/file.txt

A simplified version is:

lookup_dir(inode, path) {
    name, rest = split(path, '/');

    next_inode_no = scan_dir(inode, name);
    next_inode = load_inode(next_inode_no);

    if (empty(rest))
        return next_inode;

    if (next_inode->type == FILE) {
        errno = ENOTDIR;
        return NULL;
    }

    if (next_inode->type == SYMLINK) {
        buf = file_read(next_inode, 0, next_inode->size);
        return lookup(buf + "/" + rest);
    }

    return lookup_dir(next_inode, rest);
}

Symbolic links can form loops.

Example:

a -> b
b -> a

The kernel usually does not try to solve this with arbitrary graph-cycle detection.

Instead, it caps symlink traversal depth.

On Linux, the limit is roughly 40 symlink traversals. If the limit is exceeded, lookup fails with ELOOP.

FAT is the original DOS file system.

It is simple and uses a one-name-per-file design.

Its on-disk layout contains:

• boot sector;
• file allocation table;
• data blocks.

The boot sector stores metadata such as:

• block size;
• number of blocks;
• location of the FAT;
• location of the root directory.

The FAT itself has one entry per data block.

Each FAT entry points to the next block in the file, forming a linked list of blocks.

A directory entry stores:

• file name;
• starting block;
• metadata inline.

There are no separate Unix-style inodes.

Sequential reads are relatively straightforward.

Random access deep into a large file can be slow, because the system may need to walk the linked list.

FAT may keep multiple copies of the FAT for resilience.

Unix Fast File System and descendants such as ext2, ext3, and ext4 are inode-centric.

The layout includes:

• superblock;
• inode bitmap;
• free-block bitmap;
• inode table;
• data blocks.

The superblock stores information such as:

• block size;
• number of blocks;
• number of inodes;
• inode size;
• root inode number;
• locations of important regions.

The inode table has fixed-size inodes indexed by inode number.

The number of inodes is typically chosen when formatting the file system.

This loses some flexibility, but makes inode lookup efficient: the file system can compute the inode location with simple arithmetic.

Ext-style file systems split the disk into block groups.

Each block group contains local metadata and data regions.

This improves locality:

• an inode and its data blocks can be placed near each other;
• related metadata can be kept nearby.

It also improves resilience:

• replicated superblocks or group-local metadata can help recovery if part of the disk is damaged.

An ext inode contains metadata fields such as:

• file type;
• access permissions;
• user ID;
• group ID;
• file size;
• sector count;
• timestamps;
• hard-link reference count;
• flags and feature bits.

It also contains a block index.

Classic ext-style indexing uses:

• direct block pointers;
• one singly indirect pointer;
• one doubly indirect pointer;
• one triply indirect pointer.

Small files use direct pointers.

Large files use indirect blocks.

This lets the inode remain fixed-size while supporting large files.

File systems maintain complex on-disk data structures.

A logical operation such as creating a file may require many physical writes.

Example: creating a regular file may require:

1. finding and marking a free inode as allocated;
2. initializing the inode with type, permissions, owner, timestamps, link count, size, and block pointers;
3. finding and marking data blocks as allocated;
4. writing the file’s data blocks;
5. updating the inode to point to those blocks and record the size;
6. updating the parent directory with a name-to-inode mapping;
7. updating parent directory metadata, such as size and modification time.

These writes go to different places on disk.

Therefore the update is not naturally atomic.

Without explicit flushes or write barriers, the storage stack may reorder writes.

This is especially obvious with sophisticated storage backends such as network block devices.

A network block device may look like one disk to the OS, but internally writes may go through a network to different servers or disks.

Different writes can arrive and persist in different orders.

A flush is a request to the I/O subsystem to ensure that all previous writes have reached stable storage before continuing.

The on-disk file system can be viewed as moving through a sequence:

\[ S_1 \rightarrow S_2 \rightarrow S_3 \rightarrow \cdots \]

Between consistent states, there are transient update phases.

During normal operation, the OS can hide transient states from applications.

The problem is that a crash can happen during a transient phase.

Then the on-disk representation may be inconsistent.

Interrupted updates are a fact of life.

Examples:

• power failure;
• battery depletion;
• plug pulled;
• kernel panic;
• watchdog reset;
• machine reset;
• storage-controller firmware crash;
• controller reset;
• accidental hot-unplug of USB or external SSD;
• cable, bus, or network failure for remote storage;
• virtual machine crash;
• hypervisor crash;
• host crash;
• forced shutdown after the system stops responding.

Even if the file-system code is correct, it cannot assume that every multi-write operation will finish.

Data is the byte stream that applications asked the file system to store.

If power is lost while an application is writing a video file, that file may be damaged.

This is bad, but the damage is local to that file.

Metadata is the file system’s own structure, such as:

• superblocks;
• allocation bitmaps;
• inodes;
• directory entries;
• indirect blocks;
• extent trees;
• free-space structures.

Metadata corruption is much more dangerous.

A small metadata corruption can make the entire file system unreadable.

Therefore many file systems focus primarily on protecting metadata.

They may preserve file-system structural integrity even if the newest user data is lost or stale.

Forward recovery means finishing the incomplete update after reboot.

If the system can infer that it was trying to move from state \(S_{n-1}\) to \(S_n\), it completes the missing work and reaches \(S_n\).

Backward recovery means rolling back to the last known consistent state.

If the system cannot safely finish the operation, it restores or continues from \(S_{n-1}\).

Recent in-flight updates may be lost, but the file system becomes structurally consistent.

A major challenge is that recovery can also crash.

A correct design must tolerate:

normal operation
crash
recovery
crash
recovery
crash
...
eventual successful recovery

Recovery must not make the situation worse.

Eventually, when crashes stop, recovery should reach a consistent state.

fsck means file-system check.

The simplest approach is:

• do nothing special during normal operation;
• after a crash, scan the file system;
• find violated invariants;
• try to repair them.

This is essentially “pick up the pieces after the crash.”

A modern file system has many invariants.

After a crash, fsck sees only the final broken state.

It does not know which operation caused that state.

Different histories can produce the same broken state.

Therefore the correct repair action may be ambiguous.

Suppose the system creates file:

d/x

Steps:

1. allocate inode 17;
2. initialize inode 17;
3. add directory entry x -> 17 to directory d.

If write 3 reaches disk before write 2, and then the system crashes, fsck may observe:

• directory d points to inode 17;
• inode 17 contains old garbage or stale block pointers;
• those blocks may be marked free.

What happened?

Possibility A: a file creation was interrupted.

Possibility B: a deletion or truncation was interrupted.

The visible corrupted state may not uniquely identify the history.

So fsck cannot always know whether to keep, delete, reconnect, or repair the inode.

Traditional fsck uses conservative heuristics:

• remove impossible references;
• reconnect orphaned inodes;
• rebuild free-space maps;
• fix counters;
• quarantine suspicious files;
• put recovered files in lost+found.

This may restore structural consistency, but it can lose names, files, or recent updates, and may create spurious recovered files.

fsck may need to scan the entire file system.

On multi-terabyte disks, this can take hours.

A server that crashes may spend a long time checking disks before it becomes available again.

This is not acceptable for many modern systems.

Soft updates make fsck’s job simpler.

The key invariant is:

After any crash, the on-disk metadata must describe a structurally consistent file system.

The file system may still leak resources, such as blocks that are allocated but not reachable.

But the metadata graph should not contain dangerous impossible references.

Then fsck mainly has to clean up leaks, rather than guess the meaning of arbitrary corruption.

Soft updates explicitly track dependencies between metadata writes.

Typical rules:

1. Never write a pointer to a newly allocated object before the object has been initialized and persisted.

   Example: initialize and flush an inode before writing a directory entry that names it.

2. Never mark a resource free before all on-disk references to it have been removed.

   Example: remove old inode block pointers before marking the corresponding data blocks free for reuse.

The goal is to avoid crash states in which, for example, two files appear to own the same block.

The file system tracks metadata dependencies in the buffer cache.

Dirty metadata buffers are annotated with dependency information.

When flushing a metadata block, the system may:

• delay the write until dependencies are satisfied;
• reorder writes;
• temporarily roll back part of an in-memory block to a safe version;
• write that safe version to disk;
• later restore the in-memory version and flush the remaining changes.

This is subtle because one metadata block may contain several fields with different dependencies.

Soft updates were implemented in UFS in FreeBSD.

The lecture emphasizes that this is impressive engineering, but difficult.

Write-ahead logging, or WAL, records intended writes before applying them to the main file-system locations.

The file system writes:

• planned updates;
• commit markers.

After a crash, recovery redoes all committed writes.

This is the same basic idea as database transactions.

The lecture focuses on redo logging.

The key property is idempotence:

\[ \text{writing the same value to the same location multiple times has the same final effect as writing it once.} \]

If recovery crashes while replaying the log, replay can simply start again later.

Repeating committed writes is safe.

A simplified redo-log protocol:

1. write planned writes to the redo log;
2. flush the planned writes;
3. write a commit marker to the redo log;
4. flush the commit marker;
5. carry out the planned writes in the real file-system locations;
6. eventually free or reuse the log entries after the writes are known to be persisted.

The flush before the commit marker is essential.

Without it, the commit marker could reach disk before the log records it claims to commit.

Then recovery might see a committed transaction whose contents are incomplete.

Recovery does the following:

1. scan the log from the last known checkpoint;
2. identify complete transactions;
3. verify records, checksums, and commit markers;
4. ignore incomplete transactions without commit markers;
5. redo committed transactions;
6. advance the checkpoint after replayed writes are persisted;
7. reuse old log space.

If recovery crashes, the same committed transactions can be replayed again.

Idempotence makes this safe.

Suppose the system creates an empty file /d/x using inode 17.

The intended updates include:

• update inode bitmap: inode 17 is now used;
• initialize inode 17;
• update directory d: add x -> 17.

With WAL:

1. write these intended updates to the log;
2. flush the log records;
3. write and flush the commit marker;
4. later copy the changes from the log to their home locations.

Before the commit marker, the operation is considered not to have happened.

After the commit marker, recovery must eventually redo the operation.

Many journaling file systems log only metadata.

This ensures file-system structural consistency, but does not guarantee that the newest user data reached disk.

If user data is also logged, the guarantee is stronger, but write traffic increases significantly.

Applications with important data, such as databases, often implement their own WAL at the application layer.

For example, SQLite can protect its own data transactions.

The file system protects metadata integrity, while the application protects application-level data integrity.

• Principled and easier to reason about than fsck heuristics.
• Independent of specific metadata structures.
• Recovery only scans the log, not the whole file system.
• Recovery is deterministic and bounded.
• Natural batching: many metadata updates can be committed together.
• Sequential log writes are efficient.
• The log can be placed on a separate fast device.
• Crash during recovery is easy to handle because redo is idempotent.

• Write amplification: metadata, and possibly data, is written first to the log and then to its final location.
• The log can become a bottleneck.
• The log can fill up, forcing old transactions to be checkpointed before more writes proceed.
• Commit latency increases because flushes are required.
• Metadata-only logging does not protect newest user data.
• Full-data logging gives stronger guarantees but costs much more I/O bandwidth.

WAL is the basis of journaling file systems.

Examples:

• ext3;
• ext4;
• XFS;
• NTFS;
• HFS+ with journaling.

The tradeoff is usually attractive: extra writes during normal operation, but much faster and less ambiguous recovery after a crash.

Copy-on-write, or COW, avoids in-place modification.

The core idea is:

If we never overwrite existing blocks, we cannot leave those blocks half-updated.

Instead of changing a block in place:

1. copy the block;
2. modify the copy;
3. update parent pointers to point to the new copy;
4. repeat up to the root;
5. atomically publish the new root.

This is analogous to immutable trees in functional programming.

A COW file system is organized as a tree of blocks.

Blocks may contain:

• metadata;
• user data;
• pointers to other blocks.

The on-disk file system has:

• many blocks;
• one or more root pointers indicating the current version.

The root pointer defines the currently visible file-system state.

To modify a leaf block:

1. allocate a new block;
2. copy the old leaf block and apply the change;
3. copy the parent so it points to the new child;
4. continue copying ancestors up to the root;
5. write a new root block;
6. atomically switch the root pointer to the new root.

Old blocks remain intact until no live root or snapshot references them.

Suppose the old tree has:

• root block \(R\);
• bitmap block \(B\);
• directory block \(D\);
• other shared block \(A\).

Creating /d/x using inode 17 requires changes to:

• bitmap;
• directory;
• inode structure.

COW writes new blocks:

• \(B'\): bitmap with inode 17 allocated;
• \(D'\): directory with x -> 17;
• \(I'\): new inode 17;
• \(R'\): new root pointing to the updated blocks.

Before the root switch, these new blocks may exist on disk but are unreachable.

After the root pointer switches to \(R'\), the new file-system version is published atomically.

Recovery is simple:

1. read the small set of possible root pointers;
2. choose the newest valid root using generation numbers and checksums;
3. ignore blocks not reachable from that root.

This is backward recovery: if the new root was not fully published, the system continues from the old root.

Conceptually, one root pointer is enough.

In practice, file systems often keep several root pointers.

Reasons:

• redundancy against bit flips or broken writes;
• ability to check consistency with checksums;
• generation numbers identify the newest valid root;
• updates can rotate among root-pointer locations to reduce hot spots.

The lecture distinguishes crash interruption from another class of failures: broken writes or bit flips.

A storage device may claim to have written data, but reading it back may reveal corruption.

Modern COW file systems often use checksums to detect such corruption.

A naive COW model is sequential: one update publishes one new root.

Real systems batch many updates into one root switch.

For example, a file system may collect updates for a short time and publish them together.

This improves throughput but means more recent updates may be lost if a crash happens before the next root switch.

COW is used in several modern file systems:

• ZFS;
• Btrfs;
• APFS;
• WAFL in NetApp appliances.

COW works especially well with SSDs because SSDs tolerate random access better than spinning disks.

Linux F2FS, the Flash-Friendly File System, also uses log-structured or COW-like techniques in its write path to fit flash storage behavior.

Early operating systems often supported one file system.

Later, systems needed to support many file systems for:

• interoperability;
• removable media;
• CD-ROMs;
• network file systems;
• different workloads;
• specialization.

It would be bad to expose separate system calls such as:

read_ext4(...)
read_fat(...)
read_nfs(...)

Applications should just call:

read(fd, buf, n)

The VFS is the abstraction layer that makes this possible.

The VFS acts as:

1. a translation layer;
2. a dispatch layer;
3. a manager of the global namespace.

It maintains the Unix-style namespace rooted at:

/

It tracks:

• mount points;
• path lookup;
• open files;
• directories;
• permissions;
• metadata;
• file descriptors;
• common kernel file objects.

It maps generic operations to file-system-specific code.

Examples of concrete file systems:

• ext4;
• XFS;
• tmpfs;
• NFS;
• procfs;
• sysfs;
• device files under /dev.

The VFS is not only for convenience.

It avoids duplicated implementations of common semantics.

For example, permission checking should not be independently reimplemented in every file-system driver.

If 17 file systems each implement security checks separately, one of them will likely get it wrong.

So the VFS centralizes common behavior, while file-system drivers handle storage-layout-specific details.

Examples of file-system-specific details:

• journaling;
• COW updates;
• block allocation;
• on-disk directory format;
• persistence rules;
• caching and locking details.

From the application’s point of view:

read(fd, buf, n)
stat(path, &info)
mkdir(path)
rename(old, new)
unlink(path)

look the same regardless of the underlying file system.

The file may be:

• on an ext4 partition on a local SSD;
• in tmpfs in memory;
• on an NFS remote server;
• generated by procfs;
• backed by a device driver.

The application does not need to know.

The VFS hides many details, but not all file systems support the same metadata.

Example:

• FAT / DOS-style file systems do not store Unix users and permissions.
• Modern file systems may support extended attributes.
• Copying files with tools unaware of extended attributes may lose metadata.

So the abstraction works best for common operations such as names and byte streams, but file-system-specific features can still leak through.

DOS / Windows traditionally expose file-system boundaries using drive letters:

C:
D:
E:

Historically, A: and B: were floppy drives, while C: became the main hard drive.

Unix-like systems instead use one global namespace rooted at /.

Different file systems are attached into this namespace using mount points.

Mounting means exposing one file system at a directory inside an already existing namespace.

Example:

Before mounting:

/
├── bin
├── home
└── mnt
    └── usb

Suppose a USB file system has:

/
├── photos
└── report

Mount it at:

/mnt/usb

After mounting:

/mnt/usb/photos
/mnt/usb/report

When path lookup reaches /mnt/usb, the VFS switches from the parent file system to the mounted file system.

Applications continue to use ordinary paths.

This makes file-system boundaries mostly transparent.

The VFS can dispatch operations to in-kernel file-system drivers.

FUSE extends this idea by forwarding file-system operations to a user-space daemon.

The path still looks like an ordinary mounted file system.

But operations such as:

• lookup;
• getattr;
• read;
• write;
• readdir;

are handled by a normal user-space process.

A typical flow:

1. application calls read("/mnt/remote/a");
2. VFS resolves the path;
3. VFS sees that /mnt/remote is a FUSE mount;
4. the FUSE kernel module forwards the request to a FUSE daemon;
5. the daemon handles the request using libraries, network APIs, databases, cloud APIs, SSH, archives, etc.;
6. the daemon replies through the kernel module;
7. the application receives the result as if it came from a normal file system.

FUSE is useful because:

• development is easier in user space;
• bugs usually crash only the daemon, not the kernel;
• complex libraries can be used;
• research prototypes are easier to build;
• teaching file systems can be implemented without writing kernel modules.

FUSE has overhead:

• extra context switches;
• copying between kernel and user space;
• daemon availability matters;
• performance may be lower than in-kernel file systems.

Common FUSE-style uses include:

• sshfs: expose a remote directory over SSH as a local directory;
• cloud/object-store file systems, such as exposing buckets as files;
• archive file systems, such as browsing ZIP, TAR, or ISO contents without unpacking;
• encrypted file systems;
• research and teaching file systems.

The kernel needs to expose many kinds of information:

• processors;
• processes;
• network connections;
• storage devices;
• serial ports;
• kernel parameters.

A bad design would add a separate system call for every kind of information.

Unix-like systems instead reuse the file interface.

Pseudo file systems expose kernel information as virtual files.

Examples:

• /proc exposes process and kernel information;
• /sys exposes system and device information;
• /dev exposes device nodes.

These are not ordinary persistent files, but they use the same path and file-descriptor machinery.

Many kernel-managed objects can also be treated through the file interface:

• pipes;
• sockets;
• devices;
• terminals;
• timers in some systems;
• pseudo-files;
• kernel data objects.

The common interface is:

open(path)
read(fd)
write(fd)
close(fd)

Not every object supports every operation.

For example, a directory may not support normal writing, and a socket has different semantics from a disk file.

But the shared descriptor machinery simplifies the OS interface.

The phrase “everything is a file” is a useful Unix mantra, but not perfectly literal.

Some abstractions still have special APIs.

For example, sockets have additional socket-specific operations.

The important idea is that many resources can share one naming and access model.