How does the Java I/O bottom work?

This blog post focuses on how I/O works at the bottom. Readers of this article are eager to understand how Java I/O operations are mapped at the machine level and what the application runtime hardware does. Assume that you are familiar with basic I/O operations, such as reading and writing files through the Java I/O API. The content is not covered in this article.

Directory

• Cache processing and kernel vs user space

• Virtual memory

• Memory paging

• File-and block-oriented I/O

• File lock

• Stream-oriented I/O

Cache processing and kernel vs user space

The buffering and buffering process is the basis for all I/O operations. The term "input and output" is meaningful only for moving data into and out of the cache. Keep it in mind at all times. Typically, the process performs an I/O request for the operating system to include data from the buffer (write operations) and data fill buffers (read operations). This is the overall concept of I/O. The mechanism for performing these transport operations inside the operating system can be very complex, but conceptually simple. We will discuss it in a small part of the text.

Shows a simplified "logic" diagram that represents how block data is moved from an external source, such as a disk, to a process's storage area (such as RAM). First, the process requires its buffering to fill up through the read () system call. This system call causes the kernel to issue a command to the disk control hardware to fetch the data from the disk. The disk controller writes data directly to the kernel's memory buffer via DMA, without the need for the main CPU for further assistance. When the read () operation is requested, once the disk controller completes the cached fill, the kernel copies the data from the temporary cache of the kernel space to the process-specified cache.

It is important to note that when the kernel attempts to cache and prefetch data, the data requested by the process in kernel space may be ready. If so, the data requested by the process is copied. If the data is not available, the process is suspended. The kernel reads the data into memory.

Virtual memory

You may have heard of virtual memory many times. Let me introduce you a little bit more.

Virtual memory is used by all modern operating systems. Virtual memory means a manual or virtual address instead of a physical (hardware RAM) memory address. Virtual addresses have two important advantages:

1. Multiple virtual addresses can be mapped to the same physical address.

2. A virtual address space can be larger than the actual available hardware memory.

In the above introduction, it appears that additional work is added from the kernel space copy to the end user cache. Why not tell the disk controller to send data directly to the cache of user space? Well, this is implemented by virtual memory. Use the above Advantage 1.

By mapping the kernel-space address to the same physical address as the virtual address of a user space, the DMA hardware (which only accesses the physical memory address) can populate the cache. This cache is also visible to both the kernel and user space processes.

This eliminates the copy between the kernel and the user space, but requires the same page alignment for the kernel and user buffers. The buffer must use a multiple of the block size of the disk controller (typically a 512-byte disk sector). The operating system divides its memory address space into pages, which are fixed-size byte groups. These memory pages are always a multiple of the disk block size and typically twice times (simplified addressing). Typical memory page sizes are 1024, 2048, and 4096 bytes. The virtual and physical memory page sizes are always the same.

Memory paging

The 2nd advantage of supporting virtual memory (having addressable space larger than physical memory) requires virtual memory paging (often called a paging). This mechanism provides space for other virtual pages to be put into physical memory by using pages of virtual memory space to persist on external disk storage. Essentially, physical memory serves as a cache of paged areas. The paging area is the space on the disk where the contents of the memory pages are forced to be swapped out of physical memory.

Resizing a memory page is a multiple of the size of the disk block, allowing the kernel to send instructions directly to disk controller hardware, write memory pages to disk, or reload when needed. It turns out that all disk I/O operations are done at the page level. This is the only way that data moves between the disk and the physical memory on a modern paging operating system.

The modern CPU contains a subsystem called the Memory Management Unit (MMU). This device is logically located between the CPU and the physical memory. It contains mapping information that translates from the virtual address to the physical memory address. When the CPU refers to a memory location, the MMU determines which pages need to reside (usually by shifting or masking some bits of the address) and converting the virtual page number to the physical page number (which is implemented by hardware, which is very fast).

File-oriented, block I/O

File I/O always occurs in the context switch of the file system. The file system is completely different from the disk. The disk stores data in segments of 512 bytes per segment. It is a hardware device that knows nothing about the saved file semantics. They simply provide a certain number of slots that can hold data. In this regard, a segment of a disk is similar to memory paging. They all have a uniform size and are a large addressable array.

On the other hand, file systems are higher-level abstractions. A file system is a special way to arrange and translate data for a saved disk (or other randomly accessible, block-oriented device). The code you write almost always interacts with the file system without interacting directly with the disk. The file system defines abstractions such as file names, paths, files, file attributes, and so on.

A file system organization (on a hard disk) a series of evenly sized blocks of data. Some blocks store meta information, such as the mapping of free blocks, directories, indexes, and so on. The other blocks contain the actual file data. The meta information for a single file describes which blocks contain file data, where data ends, last update time, and so on. When a user process sends a request to read file data, the file system implements exactly where the data is located on the disk. Then take action to put these disk sectors into memory.

The file system also has the concept of a page, which may be of the same size as a basic memory page or a multiple of it. A typical file system page size ranges from 2048 to 8192 bytes, and is always a multiple of the base memory size.

Paging File system execution I/O can be attributed to the following logical steps:

1. Determines which file system paging (a collection of disk segments) the request spans. The file contents and metadata on the disk may be distributed across multiple file system pages, which may be disjoint.

2. Allocate enough kernel space memory pages to save the same file system page.

3. Establish the mapping of these memory paging to the file system paging on disk.

4. A paging error is generated for each memory page.

5. The virtual memory system is stuck with paging errors and dispatches Pagins (paging in) to validate the pages by reading the contents from disk.

6. Once Pageins is complete, the file system decomposes the original data to extract the requested file contents or attribute information.

It is important to note that this filesystem data will be cached like any other memory page. In subsequent I/O requests, some data or all file data is still stored in physical memory and can be reused directly without the need to reread from disk.

File lock

File locking is a mechanism by which a process can prevent other processes from accessing a file or restricting other processes from accessing it. Although called "File lock", it means locking the entire file (often done). Locking can usually be at a finer level of granularity. As the granularity drops to the byte level, the area of the file is usually locked. A lock is associated with a specific file, starting at the specified byte location of the file and running to the specified byte range. This is important because it allows multiple processes to collaborate on access to specific areas of a file without interfering with other processes operating elsewhere in the file.

There are two forms of file locks: shared and exclusive. Multiple shared locks can be active in the same file area at the same time. On the other hand, an exclusive lock requires no other locks to be valid for the requested zone.

Stream I/O

Not all I/O is block-oriented. There is also stream I/O, which is the prototype of the pipeline and must sequentially access the bytes of the I/O data stream. Common data flows are TTY (console) devices, print ports, and network connections.

Data flow is often but not necessarily slower than block devices, providing intermittent input. Most operating systems allow non-blocking mode to work. Allows a process to check that the input to the data stream is available and does not have to be blocked when it is not available. This management allows the process to process when the input arrives and can perform additional functions when the input stream is idle.

Further than non-blocking mode is the conditional selection (readiness selection). It is similar to non-blocking mode (and is usually based on non-blocking mode), but reduces the burden of the operating system checking flow readiness. The operating system can be told to observe the flow collection and return to the process which flow-ready instructions. This capability allows a process to reuse multiple active streams using common code and a single thread by leveraging the readiness information returned by the operating system. This approach is widely used in Web servers to handle a large number of network connections. Preparing for selection is critical for large capacity expansion.

So far, there's a lot of technical jargon for this very complicated topic.

How does the Java I/O bottom work?