Linux administrator Manual (4)-memory management

This chapter describes the memory management features of Linux, namely, virtual memory and disk cache. Describes the things, jobs, and objectives that system administrators should consider.

What is virtual memory?

Linux supports Virtual Memory, that is, the disk is used as the Ram Extension, so that the available memory can be effectively expanded accordingly. The core stores unused memory blocks on the hard disk to free up memory for other purposes. When the original content is used again, it is read back to the memory. This is transparent to users:ProgramYou can only see a large amount of available memory, but cannot decide which part is on the disk. Of course, the read/write hard disk is slower than the real memory, so the program runs slowly. This part of the hard disk used for virtual memory is called swap space.

Linux can use common files in the file system or separate partitions as swap spaces. Swap partition is faster, but swap file is easier to change the size (no need to re-partition the hard disk ). If you know how much swap space you want, you should swap the partition. If you cannot confirm, you can swap the file first, and then create swap partitions Based on the required space after a period of time.

Linux allows multiple swap partitions and/or swap files at the same time. That is, if you occasionally need more swap space, you can create an additional swap file at any time.

Generate swap space

Swap files are common files and have nothing special to the core. The only difference is that it has no holes and is prepared with mkswap. It must be on a local disk and cannot be in a file system mounted through NFS.

Holes are important. Swap files keep the disk space, so that the core can quickly swap out a page without having to go through all the operations such as locating the disk sector. The core only uses all the slices allocated to the file. Because the holes in the file mean that no disk sector is allocated for this position in the file, this is not good for the core usage.

A good way to generate swap files without holes is to run the following command:

$ Dd If =/dev/Zero of =/extra-swap BS = 1024 COUNT = 1024
1024 + 0 records in
1024 + 0 records out
$
/Extra-swap is the swap file name, the size is given by Count =. The size is preferably a multiple of 4, because the memory page written by the core is 4 kb. If it is not a multiple of 4, the last few Kb will be unavailable.

There is nothing special about swap partitions. It is generated just like other partitions; the only difference is that it is used as the original partition, that is, there is no file system, it is best to mark the swap partition as type 82 (Linux swap ), although this does not affect the core, it makes the partition list clearer.

After a swap file or swap partition is generated, you need to write a tag to use it, which includes some management information required by the core. The command is mkswap. Its usage is as follows:

$ Mkswap/extra-swap 1024
Setting up swapspace, size = 1044480 bytes
$

Note that swap space is useless now. It exists, but the core does not use it to provide virtual memory.

Be careful when using mkswap because it does not check whether the file or partition is used by other things. You may use mkswap to easily overwrite important files and partitions! Fortunately, you only need to use mkswap when installing the system.

Linux memory management limits each swap space to about 127 MB (for technical reasons, the actual limit is 127.6875 MB ). You can use up to 16 swap spaces at the same time, totaling about 2 GB.
Use swap space

Use Swapon to make an initial swap space available. This command indicates that the core swap space can be used. The swap space Path is used as the parameter. To start a temporary swap file, run the following command:

$ Swapon/extra-swap
$

If the swap space is included in/etc/fstab, it can be used automatically.

/Dev/hda8 none swap SW 0 0
/Swapfile none swap SW 0 0

Run the Swapon-a command to start all swap spaces listed in/etc/fstab. Therefore, the Swapon command is used only when extra swap space is started.
You can use free to monitor the use of swap space. It will provide all swap spaces used.

$ Free
Total used free shared Buffers
Mem: 15152 14896 256 12404
-/+ Buffers: 12368 2784
Swap: 32452 6684 25768
$

The previous row output (MEM :) displays the physical memory. The total column does not display the core physical memory (usually about 1 MB ). The used column shows the total memory used (the second row does not count as a buffer ). The free column shows all unused memory. The shared column displays the total memory shared by multiple processes. The buffers column displays the current disk cache size.
The information displayed in the next line (SWAP :) is similar to that shown above. If the behavior is 0, no swap space is used.

The same information can be obtained through top or using the/proc/meminfo file of the proc file system. It is still difficult to obtain the use information of a swap space.

Swapoff can be used to cancel swap space. Generally, this is not necessary unless it is a temporary swap space. Pages used in swap space are swapped in (Swap-> RAM). If there is not enough physical memory, the pages are swapped out (Ram-> swap, to other SWAP spaces ). If there is not enough virtual memory to put on all pages, Linux will begin to oscillate (thrash); it should be able to recover after a long time, but the system is unavailable at this time. Before canceling a swap space, check (for example, using free) whether there is enough physical memory.

All swap spaces automatically used by Swapon-A can be canceled by swapoff-. It checks the file/etc/fstab to find out what to cancel. Any manual swap space will still be used.

Even if there is a lot of idle physical memory, sometimes a lot of swap space is used. This is because a SWAp is required at a certain time, but a large process that occupies a large amount of physical memory then terminates and releases the memory. They are not automatically swapped in until the converted data is used. You don't have to worry about this situation, but you are more comfortable knowing why it happens.

Share swap space with other operating systems

Many operating systems have built-in virtual memory. Because they only need to be used at runtime, that is, they will not run at the same time, so all the other swap space is wasted except for the currently running ones. If they share the same swap space, it will be more effective. This is possible, but some hacking work is required. Tips-howto provides some advice on how to accomplish this task.

Allocate swap space

Someone may tell you that we should allocate 2 times the swap space of physical memory, but this is a false rule. The following describes how to make sure that:

Estimate your total memory requirements. This is the largest amount you may need, that is, the sum of the memory requirements required for all programs you want to run simultaneously. You can run all programs that you may run at the same time.

For example, if you want to run X, you need to allocate 8 MB to it. GCC requires several Mb (some files may need a large number of times, dozens of MB, but generally about 4 MB. The core itself uses 1 MB, and shell and some gadgets may require several hundred KB (or, a total of 1 MB ). You don't have to be too precise, just make a rough estimate, but you can be more pessimistic.

Remember that if multiple people will use the system at the same time, they will all consume memory. If two people run the same program at the same time, the total memory consumption is generally not doubled becauseCodePages and shared libraries are single.

The free and PS commands are useful for estimating memory requirements.

The first step is to add some security measures. Because the estimation of the program size is likely to be incorrect, because you may forget some programs to run and are sure you have some extra space. It should be several Mb. (It is better to allocate too much swap space than to allocate too little space, but you don't have to worry too much, because unused swap space is a waste. See the following article: about adding swap space .) Also, since it is nicer to deal with even numbers, you can round the value up to the next full megabyte.

Based on the above calculation, you know the total amount of memory you need. The actual physical memory is reduced, that is, the swap space. (In some versions of UNIX, you still need to allocate the image space of the physical memory, so the space required for calculation in the second release cannot be reduced)

If you calculate a much larger swap space than your actual physical memory (more than several times), you may need more physical memory, otherwise the system performance will be too low.

Even if the calculation shows that you do not need to swap space, it is better to have at least some. Linux uses swap space in an aggressive manner, so as to maintain a certain amount of free physical memory. Even if the memory is not required by the program, Linux will swap out some unused memory pages, in this way, you can avoid waiting for swap-that is, swap can be completed early when the hard disk is idle.

Swap space can be divided into several hard disks, which sometimes improves performance, depending on the relative speed and access mode of these disks. You can try a few solutions, but it is very difficult to know how to try it correctly. Do not trust the determination that a scheme is better than other schemes, because it will not always be right.

High-speed cache

Compared with the (real) memory, reading from a disk is slow. In addition, it is common to read the same part of the hard disk multiple times at a relatively short end. For example, you may have read an email first, then read it into the editor when you reply, and then copied it to a folder and read it with the email program. Or, consider how frequently the LS command may be used by many users on the system. Read the information only once from the disk and keep it in the hard disk. It is faster than the first read. This is called Disk buffering, and the memory used for this purpose is called buffer cache.

Unfortunately, the buffer cache is generally not large enough because the memory is limited and the resources are insufficient (as big as the data that can be used by everyone ). When the cache is full, data that is not used for the longest time will be discarded, and the memory will be released to the latest data.

Disk buffering is also used for write operations. The data to be written is often read immediately (for example,Source codeFiles are stored in the file and read by the compiler), so it is a good idea to put the data to be written in the buffer. In addition, if you only store data in a cache and do not immediately write data to a disk, write operations can be performed faster. The write operation can be completed in the background without reducing the speed of other programs.

Many operating systems have buffer caches (even if the names are different), but not all of them follow the above principles. Some are write-through: the data is immediately written to the disk (of course also written to the cache at the same time) the cache that is not written immediately is called write-back. Write-back is more effective than write-through, but more error-prone: if the system crashes, the power supply suddenly powers down, or the floppy disk is taken out before cache write-back, the changed data in the cache will be lost. This may mean that the file system is not in full working order, probably because unwritten data contains important changes in system record information.

Therefore, do not directly turn off the power without going through the normal Shutdown Process (see chapter 6), or remove the floppy disk without unmount (if it is mounted ), or what program is still using a floppy disk, or the floppy lamp is still flashing. The sync command refresh the buffer, that is, to forcibly write all the unwritten data back to the disk. If you want to ensure the security of all the data, you can use it. In traditional Unix systems, an update program runs in the background and runs sync every 30 seconds. Therefore, you do not need to use sync. In Linux, there is another Hou program bdflush, which overcomes the problem that sync sometimes suffers from sudden system crashes due to heavy disk I/O loads (due to frequent operations.

In Linux, bdflush is started by update. Generally, you do not need to consider it. However, if bdflush occasionally dies for some reason, the core will give a warning. In this case, you should manually start it (/sbin/update ).

The cache does not actually buffer files, but blocks, that is, the minimum disk I/O unit (in Linux, usually 1 kb ). In this way, all directories, Super blocks, other file system record data, and non-File System disks can be buffered.

The cache effect depends on its size. A small cache is almost useless; it can only cache a small amount of data, but may be cleared before being reused. The size depends on how much data is read and written, and the same Data Access frequency. The only method is experiment.

If the cache size is fixed, it should not be too large. Otherwise, swap will be used because the idle memory space is too small (also very slow ). To use the actual memory most effectively, Linux automatically uses all idle memory as the buffer cache. When the program requires more memory, the cache is automatically reduced.

In Linux, it does not need to do any work for Cache Usage. It is completely automatic. You do not need to worry about cache unless you need to properly shut down the system and retrieve the floppy disk.