When we talk about how information is stored and moved within a computer, we usually only think about the hard drive or RAM. But underneath it all, there's a whole system. storage architecture with tiers, caches, file systems, and partitions This determines how quickly and securely we work with data. Understanding this anatomy is key if you study systems, manage teams, or simply want to know what happens when you save a file or a disk becomes corrupted.
Throughout this article we will go through, step by step, how is storage organized from the CPU to secondary storageThis course covers: the role of cache memory, how the file system cache works in Windows, what a file is, how directories are organized, the different types of file systems (FAT, NTFS, ext, APFS, etc.), and how to partition hard drives using MBR. All with a practical approach and real-world examples.
General anatomy of storage in a computer system
In any modern computer we find several levels of memory, each with different speed, capacity and functionFrom top to bottom, we can organize them like this:
- Records! processor.
- Cache (L1, L2, L3) integrated or close to the CPU.
- Principal memory (RAM).
- Secondary memory (hard drives, SSDs, etc.).
- Auxiliary memory (USB, external hard drives, optical drives, etc.).
Registers are an extremely fast memory that can only hold one or a few pieces of data per register. They function as internal processor variablesFor example, the Program Counter (PC) stores the address of the next instruction to be executed and changes with each step. They are identified by name (PC, RI, AX, etc.), are volatile, and are based on semiconductor technology.
The main memory (RAM) consists of equal-sized cells called words (32, 64, or 128 bits, depending on the architecture). To access a memory cell, its address is used, which is transmitted via the address bus. Like registers, RAM is volatile: if the power is cut off, its contents are lost. Even so, it has much greater capacity than registers and is where programs are executed and data is loaded at runtime.
Cache memory and cached data: speeding up access
Between the CPU and main memory we find the cache memory, designed to bring the most frequently used data closer to the processor and reduce latency. It also uses semiconductor technology and is divided into cells, but it is faster than RAM and considerably more expensive, so its capacity is lower.
The idea behind caching is always the same, whether we're talking about CPU cache, disk cache, or browser cache: temporarily store information that has already been consulted, so that if it is needed again, it can be retrieved from a faster location than the original source (disk, network, etc.).
The concept of cached data This also applies at the application, server, and web browser levels. A browser, for example, caches images, scripts, CSS, and HTML files the first time you visit a page. This way, when you return to that site, it doesn't need to download everything again; instead, it reuses the local data and only requests what has changed, significantly improving loading times.
This temporary storage mechanism is also used in other components: DNS servers cache domain name records To make resolutions faster, CDNs (content delivery networks) replicate and cache static content on nodes distributed around the world to reduce latency to end users.
Secondary memory: block and sector storage
The main memory is fast but has two problems: It is volatile and does not have sufficient capacity. to store all the information we handle (documents, databases, videos, backups, etc.). That's why we need secondary and auxiliary storage: hard drives, SSDs and standards like UFSUSB drives, optical discs, etc. They are persistent, cheaper per unit of storage, and have large capacity.
These devices are not organized into small cells like RAM, but in physical blocks called sectorsTraditionally, the typical sector size has been 512 bytes. Read and write operations are performed at least at the sector level: you cannot read or write an amount of data smaller than a sector, nor can you address a fraction of a sector.
With the increase in disk capacity, it became necessary to group several sectors to better manage space. This is how sectors came about. clusters or allocation unitsThese are groups of sectors that the file system uses as the smallest unit of file allocation. This simplifies management but has implications for wasted space, as we will see later.
Files and file system: the logic about the disk
In order to take advantage of storage devices, the operating system needs a logical layer that defines how information is stored, organized, and controlled in those physical blocks. That layer is the file system.
From a logical point of view, the file system relies on two basic concepts: the archive and folder or directoryA file is a sequence of bytes stored on an external medium that is considered a logical unit: a PDF document, an image, a video, a database, etc. Each file has a name and, usually, an extension that helps to identify the type of content (for example, .png, .txt, .pdf).
A folder or directory is simply a way of group files and other directories according to the user's or system's criteria to facilitate organization. Internally, for the file system itself, a directory is simply a special file that stores information about what elements it contains and how they are structured.
The file system handles critical tasks such as assign sectors and clusters to the filesMaintain control of which blocks belong to each file, offer creation, renaming, modification and deletion operations, maintain the hierarchical directory structure, control which sectors are free and which sectors are occupied, and manage permissions and data access security.
It is common for each operating system family to have its own native file systems, although there is increasing cross-compatibility or support through specific tools, including ways to access cloud storage services. Data received from a communication line or a special device They can also be exposed as files, even though they are actually generated dynamically (devices in /dev in GNU/Linux, for example).
Attributes, permissions, and ACLs in the file system
In order for the operating system to control who accesses what, each resource (files, directories, printers, network resources, etc.) usually has associated with an access control list (ACL)This list shows the users or groups that can access it and with what permissions: read, write, execute, etc.
In addition to permissions, a file may have attributes that describe its nature or its special treatment From the system's perspective: system file, hidden, read-only, encrypted, directory, temporary, etc. These attributes influence how the file is displayed to the user and what operations are permitted.
Routes, names and wildcards
Almost all modern file systems organize data hierarchically in the form of directory treeTo uniquely locate a file or directory, its path is used, which describes the path to follow within the tree: a sequence of directories separated by a special character, the last element of which is the file or folder itself.
In Unix-like systems (GNU/Linux, macOS at its core), the absolute path begins with a forward slash / representing the root directoryExample: /home/Alicia/Documents/report.odt. In Microsoft systems, the drive letter is preceded by a colon (:) and backslashes: C:\Users\Alicia\Documents\report.odt.
We can also use relative paths, which start from the current user directoryFor example, ../../Jacinto/Documentos/memoria.odt indicates “go up two levels and then down to Jacinto/Documentos/memoria.odt”. In Unix, there are special wildcards for paths: . (current directory), .. (parent directory), and ~ (user's home directory).
Operating systems offer wildcard characters to refer to one or more files without knowing their full names. The asterisk (*) represents any combination of characterseven the absence of characters; the question mark (?) represents a single character. For example, *image*.png could match My_image.png, image.png, image1.png, and image21.png, while image??.png would only match six-character names followed by .png, such as image21.png.
From the file in RAM to the file on disk
When you create a file from an application, the data initially resides only in main memory. If you open Notepad, type your name, and haven't saved yet, the information is in... memory process notepad.exeThere is no file on disk associated with that content.
The moment you click “Save” and choose a path, the operating system creates a file in the file system and assigns one or more clusters of the storage device to store that information. From then on, the system must know at all times which blocks have been reserved for that file and in what order they should be read.
One curious detail: often the logical size of the file (in bytes of data) is much smaller than actual disk space it occupiesFor example, a text file containing the word "Javier" in ASCII takes up 6 bytes of content, but on a system where the cluster size is 4 KiB, that file will occupy 4096 bytes on disk, because the cluster is the smallest assignable unit (if you are concerned about this behavior, see What to do if internal storage appears full).
System file cache in Windows
In Windows, the operating system implements a system file cache which acts as an intermediary between I/O operations and the physical disk. By default, when a program reads or writes to a file, it is actually reading or writing to memory, in a region managed by the cache manager.
When a read operation is performed, the data is first loaded from disk into a region of system memory reserved as a file cache. From there, user-mode processes copy that data to their own address space. If the same area of the file is read again laterThe system can serve it directly from the cache without accessing the disk again, greatly speeding up performance.
In write operations, data is initially stored in the system's file cache instead of being written immediately to disk. This mechanism is known as write-back rewrite cacheThe cache manager periodically empties this memory, writing the modified blocks to the physical disk and freeing up space in the cache. The timing of the emptying depends on how long the data has been in the cache and how long ago it was last accessed.
This policy of deferring disk writing has performance advantages, but it also carries a risk: if a power outage or sudden failure occurs before the cache is clearedData that has not yet been written to disk can be lost. To balance performance and reliability, Windows launches a process called the deferred writer every second, which queues a fraction of the pages awaiting writing to disk and dynamically adjusts the amount of data it writes.
There are applications (for example, antivirus software or database management systemsThey need to ensure that writes are written to disk immediately. To achieve this, Windows offers write-through mode: if a file with the FILE_FLAG_WRITE_THROUGH flag is opened, the data is written to the cache, but the cache manager pushes it to disk without delay. Additionally, features like FlushFileBuffers allow for the forced flushing of metadata and pending content.
In some massive I/O scenarios with very large blocks, caching can worsen performance. In such cases, it's possible to open files with the FILE_FLAG_NO_BUFFERING flag so that Reads and writes go directly to the physical disk, bypassing the file cache (although some metadata may still be cached).
Types of file systems and features
The most commonly used file systems are closely tied to the operating system family for which they were designed, although many have become almost de facto standards due to compatibility. Let's look at the most important ones.
Microsoft Family File Systems
In the Windows world we can distinguish mainly two large families: FAT and NTFSFAT (File Allocation Table) is a simple and very old design, while NTFS (New Technology File System) is more modern, robust and feature-rich.
FAT relies on a table that indicates for each cluster which file it belongs to and what the next cluster in the file chain is. Its most notable versions areFAT12 (early 80s, with volumes up to 32 MB and 8.3 naming conventions), FAT16 (up to about 90 GB with 32 KiB clusters), VFAT (32-bit protected mode support and long naming conventions in Windows 3.11/95), and FAT32 (introduced in Windows 95 OSR2, with a theoretical capacity of almost 8 TB but limited by Microsoft to smaller sizes, and with a maximum file size of 4 GB). Later came exFAT, optimized for USB drives and flash drives, with fewer file size limitations.
FAT has several drawbacks: Their structures easily cause fragmentation.It does not implement advanced permissions or journaling, and to be fast, the entire FAT table is usually loaded into RAM, consuming a significant amount of memory. On very large disks, if only part of the table is cached, accessing highly fragmented files can involve many disk jumps just to read FAT fragments.
NTFS, for its part, introduces performance, safety and reliability improvementsIt supports granular permissions, ACLs, encryption, compression, journaling, and automatic recovery from certain errors. It is based on HPFS (developed for OS/2) and can address enormous volumes (up to tens of terabytes in typical implementations). Several versions have been released: from NT 3.1 (v1.0) to the 3.x versions used from Windows 2000 and XP onward.
File systems in Apple
Apple computers have gone through several generations of file systems, each adapted to the technology of the time. The most representative are: MFS, HFS, HFS+ and APFS.
MFS (Macintosh File System) was the first file system for the original Macintosh in 1984. It allowed filenames of up to 255 characters, although it only indexed the first 63, and handled volumes up to 256 MB. As storage capacities grew rapidly, it was soon replaced by HFS (Hierarchical File System), also known as Mac OS Standard, which introduced a real hierarchical organization and supported volumes up to 2 TB (with individual files up to 2 GB).
Later came HFS Plus (HFS+), or Mac OS Extended, with Support for journaling, Unicode names, and volumes up to 8 exabytesIt has been the primary file system on Macs for many years. With the advent of SSDs and the need for modern encryption and snapshots, Apple designed APFS (Apple File System), specifically optimized for flash memory and supported since macOS Sierra and iOS 10.3. APFS improves space management, file cloning, and performance in intensive operations.
File systems in GNU/Linux and Unix
GNU/Linux supports a wide variety of file systems, but the most common are the families ext (ext2, ext3, ext4), ReiserFS, XFS and ZFS (the latter through external modules).
ext2 was introduced in 1993 as an improved successor to ext, with support for volumes up to 16 TB and files up to 2 TB, and 256-character filenames. ext3, released in 2001, added journaling and relied on more efficient structures such as balanced binary trees, while maintaining compatibility with ext2. ext4, released in 2006, improved speed, CPU usage, and increased maximum capacities to 1 exabyte for volumes and files up to 16 TB.
ReiserFS was one of the first journaling systems supported by the Linux kernel, offering good performance with small files. XFS, originally from IRIX and ported to Linux, also implements journaling and is notable for its maximum volume size and performance in high-performance environments, capable of handling up to 16 exabytes.
ZFS, originally developed by Sun Microsystems for Solaris, introduces a combined file system and volume manager approach. It allows files up to 16 exabytes and gigantic volumes, with advanced features such as snapshots, integrity verification using checksums, self-testing, and automatic repair of silent corruption.
How file systems address blocks: inodes and extents
In Unix-like systems, file systems such as ext2 and ext3 use a structure called inode to represent each file. The inode stores metadata (permissions, owner, size, timestamps, etc.) and a series of pointers to data blocks.
In ext3, each inode contains 12 direct pointers to data blocks, plus three indirect pointers: one single, one double, and one triple. With the direct pointers Twelve blocks of data can be accessed directly. If the block size is 4096 bytes, this is equivalent to 48 KiB of data (12 × 4096).
If the file grows beyond that limit, the simple indirect pointer comes into play, which points to a special block containing only addresses of other data blocks. Assuming 32-bit words (4 bytes), a 4096-byte block can hold 1024 addresses. This adds up to 4 MiB extra (1024 × 4096 bytes). For even larger files, double and triple indirect pointers are used, which chain together several levels of address blocks, reaching capacities of up to 4 GiB (double) and 4 TiB (triple) per file.
The sum of the capabilities provided by the 12 direct pointers, the single indirect pointer, the double and the triple pointer gives us the theoretical maximum file size in ext3: 48 KiB + 4 MiB + 4 GiB + 4 TiB, approximately.
In ext4 this structure is redesigned: pointers become 48 bits to support larger devices (up to 1 EiB) and indirection blocks are used instead. extentsAn extent describes a range of contiguous blocks using two values: the starting block and the number of blocks. The 60 bytes that ext3 used for addresses are reused in ext4 to store multiple extents and a header. If a file is highly fragmented and requires more extents, a tree (HTree) is organized, with the root stored in the inode and leaf nodes containing the additional extents.
This extents-based structure aims to reduce fragmentation and improve performancebecause it works with large contiguous ranges instead of long lists of individual blocks.
Other specific and virtual file systems
In addition to disk and network file systems, there are systems designed for very specific purposes or to function in virtual modeSome notable examples:
- swap: disk area reserved for virtual memory, where pages are swapped when RAM runs out.
- archfs: user space file system (FUSE) that allows browsing rdiff-backup repositories.
- cdfs: a virtual file system in Linux for accessing individual data or audio tracks on compact discs.
- udev and devfs: used in GNU/Linux to manage device files under /dev.
- ftpfs and nntpfs: User Space Systems (FUSE) that expose data accessible by FTP or NNTP as if it were a local file system.
FUSE (Filesystem in Userspace), included in the Linux kernel since version 2.6.14, allows non-privileged users develop and install file systems These programs run in user space, with a kernel module acting as an intermediary. This greatly increases the variety of available file systems (NTFS, EncFS, etc.), even allowing virtual disks (such as .vdi files) to be mounted as if they were physical disks. The downside is that the constant switching between kernel mode and user mode introduces some performance penalty.
Underneath, everything relies on VFS (Virtual File System), an abstraction layer that It unifies access to file systems with very different architectures.so that applications see a common interface regardless of the specific type of system underneath.
Transactions and journaling in file systems
A transaction is a sequence of operations that must be executed atomicallyEither all operations are completed successfully, or if something fails, the system must be restored to its original state. This concept applies to databases, but also to file systems through journaling.
Journaling, or diary recording, consists of keeping a journal in which the planned modifications are noted before they are applied to the actual file system. If a power outage or unexpected failure occurs while writing is in progress, the journal is checked when the system restarts: incomplete transactions are rolled back or redone depending on the operating mode, ensuring that the file system structure remains consistent and preventing serious corruption.
Partitions, MBR and logical disk organization
A hard drive or SSD is rarely used as an undivided monolithic block. The normal thing to do is to create partitionsThese are contiguous segments of the disk delimited by a start sector and an end sector. Each partition can contain a different file system, allowing, for example, Windows and Linux to be installed on the same disk, or for system data and user data to be separated.
From the operating system's point of view, each partition can be handled as an independent logical unitIn Windows, it has its own drive letter (C:, D:, etc.); in GNU/Linux, it's mounted in specific directories (/home, /var, etc.). This provides flexibility and makes tasks like reinstalling an operating system easier without touching the data partition.
In classic MBR partitioning schemes, information about the partitions is stored in the partition table stored in sector 0 of the disk, known precisely as the MBR (Master Boot Record). The MBR includes three main elements: a small boot program (bootloader, which is responsible for launching the operating system or a more complex loader), the partition table (with up to four entries) and a 16-bit “magic code”, usually 0x55AA, which indicates that the MBR is valid.
The MBR partition table only supports four primary partitionsTo overcome this limitation, two additional types were defined: extended partitions and logical partitions. An extended partition is actually a special type of primary partition that does not directly contain data, but rather It houses multiple logical partitions within itselfThus, we can have a maximum of four entries in the MBR (three primary and one extended, for example) and within the extended one as many logical entries as we need.
In practice, when a new disk does not have an initialized MBR, tools like gparted detect that There is no recognizable label or magic code 0x55AA and display warnings. The first step is usually to create a new partition table (for example, an MSDOS type for MBR). From there, primary partitions are created, and if more than four file systems are needed, an extended partition is created to accommodate additional logical partitions.
During planning, it's advisable to decide in advance what size and type each partition will have: for example, a large primary partition for data in NTFS, another in ext4 for Linux, a FAT32 for swap, and an extended partition with several logical partitions, or convert your hard drive for other uses. If the four primaries are exhausted without having reserved an extended oneWe will have to delete some partitions to rebuild the structure correctly.
In administration or training environments, it is common to use a Live distribution, such as SystemRescue, boot into graphical mode (using startx) and use tools such as gparted to create, resize and format partitions with different file systems (NTFS, FAT32, ext3, ext4, etc.), checking along the way how an MBR is initialized and how the changes are reflected in the partition table.
This whole network of registers, caches, main memory, disks, file systems, journaling, and partitions means that when you press "Save", The data travels from the CPU register to a specific block on a physical disk. passing through several layers of cache and logical structures. Understanding how all these pieces fit together allows for a better understanding of performance issues, disk errors, file corruption messages, or design decisions such as cluster size or the choice of a file system for each use case, and also opting for free cloud storage services according to the needs.
