"Turn" to uncover the Linux swap puzzle

Why Choose Linux? Because Linux allows you to master everything you do!
Why hate Windows? Because Windows lets you not know what you are doing!
That's why I like Linux. As long as I want to, I can be the bottom of the system operating mechanism to see clearly, can master everything. Windows, despite its beautiful interface, makes you wonder what she is thinking. I don't like the feeling of being aloof.


If you feel headache at the sight of this headline, or do not care about Linux's internal technology at all, then I would advise you: Don't use Linux. You're just chasing the tide, not really liking it. Linux does not have Windows handy, but it is "strong" than Windows. If you are interested in the stability of Linux, especially if you want to use Linux as a Web server, take a look below!

Swap, the swap area, how many people cared about it, in addition to installing Linux? In fact, swap adjustments are critical to the performance of Linux servers, especially Web servers. By adjusting swap, you can sometimes bypass system performance bottlenecks and save on system upgrade costs.

The contents of this article include:


Swap Fundamentals

Breakthrough 128M Swap Limit

Impact of swap configuration on performance

Swap performance Monitoring

System commands for Swap operations
Swap Fundamentals

The principle of swap is a more complex issue that needs a lot of space to illustrate. Here is a brief introduction to the details of the swap implementation in a future article.

As we all know, the modern operating system has realized the "virtual memory" this technology, not only in the function of the physical memory limit, so that the program can manipulate more than the actual physical memory space, more importantly, "virtual memory" is to isolate each process of the security network, so that each process is not affected by other programs interference.

The function of Swap space can be simply described as: when the physical memory of the system is not enough, it is necessary to release some of the space in physical memory for use by the currently running program. The freed space may come from programs that have not been manipulated for a long time, and the freed space is temporarily saved in the swap space until those programs run, recovering the saved data from the swap into memory. In this way, the system always does swap swap when physical memory is not enough.

Computer users often encounter this phenomenon. For example, when using a Windows system, you can run multiple programs at the same time, and when you switch to a program that has been ignored for a long time, you hear the hard drive "rushing" straight. This is because the memory of the program is "stolen" by those programs that are running frequently and placed in the swap area. Therefore, once the program is placed on the front end, it will retrieve its own data from the swap area, put it in memory, and then run.

It is important to note that not all data exchanged from physical memory will be put into swap (if so, the swap will be overwhelmed), and a significant portion of the data is exchanged directly to the file system. For example, some programs will open some files to read and write to the file (in fact, each program to open at least one file, that is, running the program itself), when the need to swap out the memory space of these programs, there is no need to put the file part of the data into the swap space, and can directly put it into the file. If it is a read file operation, then the memory data is released directly, do not need to swap out, because the next time you need to recover from the file system directly, if you are writing files, only need to save the changed data to a file in order to recover. However, the data for objects generated with malloc and the new function are different, they require swap space because they do not have a corresponding "reserve" file in the file system and are therefore referred to as "anonymous" (Anonymous) memory data. This type of data also includes some state and variable data in the stack. So the swap space is the swap space for "anonymous" data.

Breakthrough 128M Swap Limit

Often see some Linux (domestic Chinese version) installation manual has this description: Swap space can not exceed 128M. Why is there such a claim? Before explaining the origin of the figure "128M", give the question a reply: There is no 128M limit! Now the limit is 2g!

The swap space is paged, and the size of each page is the same as the size of the memory page, facilitating exchange of data between swap space and memory. When an older version of Linux implements swap space, the first page of the swap space is used as a bit map for all swap space pages. This means that every bit of the first page corresponds to a single page of swap space. If this is 1, the page swap is available, or 0 indicates that the page is a bad block and cannot be used. So the first swap mapping bit should be 0, because the first page swap is the mapping page. In addition, the last 10 mapping bits are also used to represent the Swap version (the original version is swap_space and the current version is Swapspace2). So, if the size of a page is S, this method of swap can manage "8 * (S-10)-1" swap pages altogether. For i386 systems s=4096, the size of the space is 133890048, if you think 1 mb=2^20 byte, the size is exactly 128M.

The reason for the management of swap space is to prevent bad blocks in the swap space. If the system checks for a bad block in the swap, Mark 0 on the corresponding bit map to indicate that the page is not available. This way, when using swap, the bad block is not used and the system generates an error.

Now the system Designers think:

1. The hard drive is now of good quality, with few bad pieces.

2. Even if there is not much, just list the bad blocks without having to create a map for each page.

3. If there are many bad blocks, this hard drive should not be used as swap space.

As a result, Linux now cancels the bit mapping method, and it also cancels the 128M limit. Directly with address access, limited to 2G.

Impact of swap configuration on performance

Allocating too much swap space wastes disk space, and there is too little swap space for the system to get an error.

If the system is running out of physical memory, the system will run slowly, but still be able to run, and if the swap space is exhausted, the system will have an error. For example, the Web server can derive multiple service processes (or threads) depending on the number of requests, and if the swap space is exhausted, the service process cannot start, and the "application is out of memory" error usually occurs, causing the deadlock of the service process to be severe. Therefore, the allocation of swap space is very important.

Typically, the swap space should be greater than or equal to the size of the physical memory, the minimum should not be less than 64M, usually the size of the swap space should be 2-2 of physical memory. 5 times times. However, depending on the application, there should be different configurations: if it is a small desktop system, you only need a small swap space, while the large server system requires different sizes of swap space depending on the situation. In particular, the database server and Web server, with the increase in traffic, the swap space requirements will also increase, the specific configuration see the description of the server products.

In addition, the number of swap partitions has a significant impact on performance. Because swap operations are disk IO operations, if there are multiple swap zones, the swap space is allocated in a rotating manner to all swaps, which greatly balances the IO load and speeds up swap. If there is only one swap area, all switching operations will make the swap area very busy, leaving the system most of the time in a waiting state and inefficient. With the performance monitoring tool you will find that the CPU is not very busy at this time, but the system is slow. This shows that the bottleneck on the IO, relying on the speed of the CPU can not solve the problem.
System performance Monitoring

While the allocation of swap space is important, performance monitoring is more valuable when the system is running. Through the Performance monitoring tool, you can check the performance indicators of the system to find the bottleneck of system performance. This article only describes some of the commands and uses that are related to swap under Solaris.

The most common is the Vmstat command (which has commands on most UNIX platforms), which allows you to see most of the performance metrics.

For example:
# Vmstat 3
Procs Memory Swap IO system CPU
R b w swpd free buff cache si so bi bo in CS us sy ID
0 0 0 0 93880 3304 19372 0 0 10 2 131 10 0 0 99
0 0 0 0 93880 3304 19372 0 0 0 0 109 8 0 0 100
0 0 0 0 93880 3304 19372 0 0 0 0 112 6 0 0 100
............

Command Description:
The parameters that follow the Vmstat specify the time interval at which the performance metrics are captured. 3 indicates a capture every three seconds. The first line of data does not have to look, it has no value, it only reflects the average performance since the boot. Starting from the second line, the system performance metrics are reflected every three seconds. Among these performance indicators, swap-related include the following:


Procs under the W
It represents the current (within three seconds) of the number of processes that need to free memory and swap out.

The SWPD under memory
It represents the size of the swap space used.

Si,so under Swap
Si represents the total amount of swap back memory (swap in) per second (within three seconds), in Kbytes;so representing the total amount of swap out memory (swap out) per second in the current (three seconds) unit of Kbytes.
The larger the number of indicators above indicates the busier the system. The performance of these indicators is related to the system's specific configuration. System administrators should be in normal operation of the system, note the values of these indicators, in the event of system problems, and then compare, will quickly find the problem, and set up the normal operation of the system standard indicator values for performance monitoring use.

In addition, the use of Swapon-s is also a simple way to see the current swap resource usage. For example:
# swapon-s
Filename Type Size used priority
/DEV/HDA9 Partition 361420 0 3

It is easy to see the size of the used and unused resources in the swap space.

The swap load should be kept below 30% to ensure good system performance.

System commands for Swap operations


Add swap space, in the following steps:
1) become a Super User
$su-root

2) Create swap file
# dd If=/dev/zero of=swapfile bs=1024 count=65536

Create a swap file that has contiguous space.

3) Activate swap file
#/usr/sbin/swapon Swapfile

Swapfile refers to the interchange file that was created in the previous step. 4) Now that the new swap file has worked, the previous steps will not be remembered after the system restarts. So to record the name of the file in the/etc/fstab file, and the swap type, such as:
/path/swapfile None Swap sw,pri=3 0 0

5) Check if the swap file is added
/usr/sbin/swapon-s


Delete the extra swap space.
1) become a Super User

2) Use the Swapoff command to retract the swap space.
#/usr/sbin/swapoff Swapfile

3) Edit the/etc/fstab file to remove the entity from this swap file.

4) Reclaim this file from the file system.
#rm Swapfile

5) Of course, if this swap space is not a file, but a partition, you need to create a new file system and then attach it to the original file system.

Advanced System Management-memory management
Memory management-swap files and RAMDisk virtual memory refers to the use of disk as a physical memory extension, so that the size of the available memory increases accordingly. The kernel writes the contents of a memory block that is temporarily unused to the disk, and then it can use that memory for other purposes. The content written to the disk may be eliminated, and it will be re-read into memory only if it is necessary to use the content. These operations are completely transparent to the user, and the program running under Linux only sees a large amount of memory available and does not notice where the memory is. Of course, reading and writing hard drives is much slower than using real memory directly, so the program is not as fast as it has been running in memory. The portion of the hard disk used as virtual memory is called swap space.
Linux can use a regular file in the file system or a separate partition as the swap space. The swap partition is faster, but the size of the swap file is easy to change, and there is no need to repartition the entire hard drive, and when you know how much swap space you need, you should use the swap partition, but if you're not sure, you can first use a swap file and then use it for a while, You can get the size of the swap space you really need, and then you'll be able to create a suitable size swap partition. Linux allows you to use several swap partitions and/or swap files at the same time, and if you only occasionally need another swap space, you can set up an additional swap file at that time instead of repartitioning to change the size of the swap partition.
Now we'll show you how to create swap space. A swap file is an ordinary file, its requirement is that it cannot have holes, and it is prepared with mkswap, so it must be on the local hard drive. How do I create a swap file? You can create a swap file that can be used with the following command:
#dd If=/dev/zero of=/swapfile bs=1024 count=2048
This creates a 2048K byte interchange file, because the size of the memory page is 4K bytes, so count is preferable to a multiple of 4 to be fully utilized. The creation of swap partitions is the same as the creation of other partitions, except that the types are different.
Once you have created the swap space, you need to use Mkswap to add some necessary initialization information to the swap space:
#mkswap/swapfile 2048
At this point the swap space is not being used by the kernel as virtual memory, it is only present. You must be very careful when using mkswap, because it does not check whether the file or partition has been used by others, so it is possible to overwrite important files and partitions.
An initialized swap space must use the command swapon command to tell the kernel that the swap space can be used. The command is as follows:
#swapon/swapfile
If you write the relevant information to/etc/fstab, you can start the system and automatically use the swap space.
#swapon –a
This command will start all the swap spaces listed in the Fstab. and
#swapoff
This command will remove the swap space initiated by the swapon–a.
To see memory usage, you can use the command:
#free
The first line of output (Mem:) shows the usage of physical memory. The total column does not show the memory used by the kernel, which typically will be nearly a megabyte. The Used column (used column) shows the sum of the used memory (the second row does not count the buffers in). The free column displays all unused memory that is not being used. Shared columns show the size of memory that is shared by several processes, and the more memory you share, the better the situation. The cache columns (buffer column) show the current disk cache size. The buffered column (cached column) shows the size of the cache that was used.
The last line (swap:) shows the information corresponding to the swap space. If the value of this line is zero, it means that your swap space is not being hit alive.
Here's a discussion of the size of the swap space: Some people will say that you need twice times the capacity of physical memory to allocate swap space, this method is not practical. The right thing to do is to estimate the maximum amount of memory you need at some point, that is, the sum of the memory required for all the programs you run at the same time, and you can do this by running all the programs at the same time. As a practical example, running the X system will need to allocate approximately 8MB of memory, and GCC requires a few megabytes, and the kernel itself uses about 1 megabytes, a normal shell, and some other tools that may require several hundred bytes. A rough estimate will give you a close-up figure, but you have to consider the worst case scenario, for example, when several people use the system at the same time, they will consume memory. The free and PS commands are useful for estimating the amount of memory required. Based on the calculations above, you'll know how much memory you'll need to sum up. So, in order to allocate swap space, you just need to subtract the actual physical memory from the total amount of storage needed, and you'll know how much swap space you need. If you calculate the swap space capacity much larger than your physical memory (more than twice times more), you usually need to buy some more memory, otherwise, the system performance will be very low.
The swap space above is to use disk space as memory, but in some cases, we need a high-speed disk space, which is exactly the opposite of the above, we can use RAMDisk to the physical memory as a disk to achieve this demand.
Using RAMDisk is a very simple thing to do. First, you need your kernel to support RAMDisk, and all you need to do is format a ramdisk and hook him up to a directory. Can be used:
#ls-al/dev/ram*
Lists all the available RAMDisk. These ramdisk do not rob memory unless a type of operation is formatted. Here is an example of using RAMDisk:
#mkdir/tmp/ramdisk
#mke2fs/dev/ram0
#mount/dev/ram0/tmp/ramdisk
These three commands will create a directory for RAMDisk, create the RAMDisk file system, and load this ramdisk to/tmp/ramdisk. Now you can use this directory as you would with other directories.
The default size of RAMDisk is 4mb=4096 blocks. You can see the size of your ramdisk when you are mke2fs. Perform df-k/dev/ram0 How much space you actually use:
#df-K/dev/ram0
This area will be refreshed when the system restarts, so do not place any data that is not copied in this area. If you have modified this directory and need to keep these changes, take some steps to back it up.
So how do you change the size of the ramdisks? You can include this line in the lilo.conf file:
ramdisk_size=10000, so that after you use the LILO command and restart the computer, the default size of RAMDisk will be 10M. Here is an example of a/etc/lilo.conf file:
Boot=/dev/hda
Map=/boot/map
install=/boot/boot.b
Prompt
timeout=10
Image=/boot/vmlinuz
Label=linux
Root=/dev/hda1
Read-only
ramdisk_size=10000
When you compile ramdisk in the form of a module, you can determine the size of the RAMDisk when loading. This can also be done by modifying the option settings of the/etc/conf.modules.
Options Rd rd_size=10000, or specify parameters to Ismod on the command line
Insmod Rd rd_size=10000
Here's how to use such a module
1) Uninstall RAMDisk,
#umount/tmp/ramdisk
2) Unload the module (which is automatically loaded in the process mentioned in the previous section),
#rmmod Rd
3) load the RAMDisk module and set its senior colonel to 20M,
#insmod Rd rd_size=20000
4) Create a file system,
#mke2fs/dev/ram0
5) Load RAMDisk,
# Mount/dev/ram0/tmp/ramdisk
In practical applications, we can attach RAMDisk to the directory of the Web server, or to the directory of the Mail service, for a quick response.

"Turn" to uncover the Linux swap puzzle