Operating System Learning Note: The structure of large-capacity memory

Introduction of large capacity memory structure
Outlines the physical structure of secondary and level three memory devices

1. Disk
Large capacity external memory.

Transfer rate: The rate of data transfer between the drive and the computer.
Positioning time: Random access time, consisting of the seek time (the time required to move the magnet arm to the desired cylinder) and the rotation wait time (the time required to wait for the desired sector to rotate to the magnetic arm).

A disk drive is connected to a computer through a set of lines called the I/O bus (IO bus). There are a variety of buses, such as Eide, ATA, SATA, USB, FC, and SCSI.

Data transfer on the bus is performed by a special processor called a controller. Divided into host controller and disk controller. The host controller is on the computer, the bus ends, and the disk controller is on disk. The computer sends a command to the host controller through a memory-mapped port, and the host controller then transmits the command to the disk controller via a message, and finally the disk controller manipulates the disk drive hardware to execute the command. Disk controllers typically have built-in caches.

2. Tape

Second, the disk structure
Can be seen as a one-dimensional logical block array. The logical block is the smallest transmission unit, usually 512B. You can set the logical block size by using low-level formatting.
A one-dimensional logical block array is mapped sequentially to the sector of the disk. Sector 0 is the first sector of the first track of the outermost cylinder, so it is arranged from outside to inside. This can theoretically convert the logical block number to an old-fashioned disk address consisting of a cylinder number + track number + sector area code. However, this conversion is not easy, because the disk will have defective sectors, the mapping must be replaced with other free sectors, followed by some disks, the number of sectors per track is not the same. The reason is that the centrifugal far track is longer and the number of sectors is more. In order to keep the disk read and write data rate constant, there are two ways: 1 is the track density uniformity, read and write the outer track, the speed is faster than the relatively internal track, this is a constant linear speed (CLV), 2 is the same speed, but the external track density is reduced, this is a constant fillet speed (CAV).

Third, disk attached
There are two ways for a computer to access disk storage, one is through an IO port or host-attached storage, and the other is a remote host through a distributed file system, called a network-attached storage.
1. Host attached storage
Accessed through the local IO port. There are several technologies: IDE, ATA, SATA, SCSI, or FC. The latter 2 kinds are commonly used in high-end workstations and servers. SCSI is a bus structure; FC is a high-speed serial structure.
A variety of storage devices are available for host-attached storage, including hard disks, RAID arrays, CDs, DVDs, and tapes.

2. Network Attached storage
A dedicated storage system for total remote access to the data network. The client accesses the NAS through a remote process invocation interface. A remote process call (RPC) can be done through TCP or UDP on the IP network. Storage units are typically used with RAID arrays with RPC interface software.

Network-attached storage for LAN (by, what about cloud computing?) ) on all computers provides a convenient way to share storage. However, efficiency and performance are inferior to host attached storage.
iSCSI is a new network-attached storage protocol that uses IP network protocols to implement SCSI protocols.

3. Storage Area Network
A storage Area network (SAN) is a private network between a server and a storage unit, using a storage protocol rather than a network protocol. The advantage lies in flexibility, where multiple hosts and storage arrays can be attached to the same SAN, and storage is dynamically assigned to the host. Sans enable a server cluster to share the same storage, or to have a storage array directly connected to multiple hosts. San ports are much lower cost than storage arrays.

FC is one of the most common San interconnects. InfiniBand is an emerging, dedicated bus architecture that provides high-speed Internet hardware and software support between servers and storage units.

Four, disk scheduling
The time to access the disk consists of two main parts: Seek time and rotation delay. The seek time is the time at which the magnetic arm moves the head to the cylinder containing the target sector, and the rotation delay is the time at which the disk needs to rotate the target sector to the head. Disk bandwidth is the transfer rate. You can schedule disk IO to improve speed and bandwidth by adjusting the access order.

Whenever a process requires disk IO, a system call is made to the operating system. If the disk and controller are idle, the request is processed immediately, otherwise it is queued to the disk request queue. This involves a disk scheduling algorithm.

1. FCFS Dispatch
First come first serve (FCFS). Fair, but not the quickest. Because the head may be too busy to handle.

2. SSTF Dispatch
Shortest seek time first algorithm. The request is processed along the road during the head movement. The algorithm may cause some requests never to be processed, because the request flows continuously, in the distance only innocently waits.

3. Scan scheduling
The magnetic arm moves from one end to the other, processing the request along the road. When you reach the other end, go back and handle it along the way. Also known as the elevator algorithm.

4. C-scan Dispatch
A variant of scan. When you move from the head to the other end, the request is processed along the road. When you reach the other end, go back, but don't deal with it, but return to the original starting point and start again.

5. Look dispatch
Scan and C-scan do not actually run from one end to the end of the other, but instead move to the furthest point of demand in a direction and go back. Returned as well. Called Look and C-look.

6, the choice of disk scheduling algorithm
Many algorithms, how to choose?
Disk Service requests are largely affected by file allocation methods. Continuously allocated file demand points are located close to each other, while linked or indexed files are physically dispersed and a large number of heads move.
In addition, disk scheduling is not only determined by the disk controller, but is also controlled by the operating system.

Five, Disk Management
1. Disk formatting
Divided into low-level formatting and formatting.
Low-level formatting, also known as physical formatting, is the function of dividing a new disk into sectors so that the disk controller can read and write. Each sector has a corresponding data structure, there will be sector numbers and error correction code. How to partition the size of a sector is a problem. Typically, the disk is shipped from the factory with low-grid processing.
And our own format, which is referred to as logical formatting. Is that the operating system writes the file system to disk. Before that, you should also partition the disk into partitions that consist of one or more cylinders.
Some operating systems or database software directly use a partition without logical partitioning, this living disk, there is no task file system data structure, only as a large sequential array.
To improve efficiency, most operating systems centralize blocks into a chunk, called a cluster. Disk IO is done through blocks, but file system IO is done through clusters.

2. Guide block
You need to run an initialization program when the computer is powered on or restarted. It initializes all aspects of the system, from the CPU registers to the device controller and memory, and then starts the operating system. For most computers, the bootstrap program is saved in the motherboard's ROM, but this is only a small part of the bootstrap program, which is further raised from the disk into a more complete bootstrap program. This part of the complete program is saved on the disk's boot block and the boot block is located in a fixed location on the disk. The disk that owns the boot partition is called the boot disk or system disk.

3. Bad block
There are more or less bad blocks per disk. There are several ways to handle these blocks. For example, the IDE controller, when the logical format, will be written in the fat entries marked that it is not available, SCSI disk maintenance of a bad block list, the use of the process is constantly updated. The controller uses a spare block to logically replace the bad block. Bad block lists and spare blocks already exist at low grid.

Vi. Exchange Space Management
Swapping refers to the movement of content between disk and memory. When physical memory is insufficient, the process moves from memory to disk to free up memory space. Of course, the swap is combined with the virtual memory technology and the swap page, eliminating the need to swap the entire process.
Because virtual memory uses disk as an extension of memory, and disks are much slower than memory, swapping space can severely affect performance. Exchange-space management is an underlying task of the operating system, which is designed to provide optimal throughput for virtual memory.

1, the use of swap space
Different operating systems are used in different ways. Some swap space is used to hold the entire process image, including code snippets and data segments, and some to store swapped-out memory pages.

2. Swap space position
Can be created on an ordinary file system, or on a separate disk partition, or simply using a raw disk. The normal file system needs to traverse the directory structure and disk allocation data, the efficiency is low, the raw disk is efficient.

3. Exchange Space Management

VII. RAID Structure
RAID, redundant array of disks.
In the past, raid was made up of many small, inexpensive disks as a replacement for large and expensive disks, and today, RAID is still in use because of its high reliability and high transfer rates.

1. Improve reliability through redundancy
such as mirroring

2. Improve performance by parallel processing
such as Stripe

3. RAID level
such as RAID0,RAID1,RAID5, etc.

Eight, stable storage implementation
The system maintains two physical copies for each logical block. The output operation is performed as follows:
1. Write the information to the first block of physical block
2, when the first write successful completion, and then write the same information on the second piece of physical block
3, only when the second write successfully completed, only to declare that the write operation completed successfully
When recovering from a mistake, each pair of physical blocks is checked. Two pieces are not wrong and the contents are the same, everything is all right; one mistake, the other is replaced with another; the two pieces are error-free but the content is different, then replace the first block with the second.
Many storage arrays add NVRAM (nonvolatile) as a cache because the completion of the wait disk write (synchronous IO) operation is quite time-consuming.

九、三级 Storage Structure
Refers to removable storage devices.

1, level three storage devices
1) Removable disk
Removable disks, discs, etc.
2) Tape
3) Future Technology

2, operating system support
The two main tasks of the operating system are to manage the hardware and provide an abstraction of the virtual machine for the application. For disks, the operating system provides two abstractions, one is the raw device and the other is the file system. So what does the operating system do when the storage media is removable?
1) Application Interface
Handle removable disks as if you were working on a fixed disk. and open the entire tape as a raw device.

2) file naming
Another issue is how to name the files on removable media. Because it is not fixed, it cannot be distinguished as a fixed disk by a name such as a drive letter + path. Modern operating systems typically do not address the namespace issues of removable media, but are determined by the application system and the user.

3) Tiered Storage management
Removable Storage is often used to make backups.

3. Performance
The three most important performance indicators for level three memory are speed, reliability, and price.

Operating System Learning Note: The structure of large-capacity memory