Use virtual memory (VM) file system and bind and install

With the release of Linux 2.4, a large number of file systems are possible, including reiserfs, XFS, gfs, and other file systems. These file systems sound really cool, but what do they actually do, what do they do well in, and how can they be safely used in a Linux product environment? Daniel Robbins answers the above questions by showing you how to create these advanced file systems in a Linux 2.4 Environment. In this section, Daniel briefly introduces tmpfs, a VM-based file system, and introduces new possibilities brought about by the "binding" installation feature of version 2.4.

In my previous articles in this series, I introduced the benefits of creating logs and using reiserfs, and showed how to install a stable Linux 2.4-based reiserfs system. In this article, we will talk about several secondary topics. First, we will briefly introduce tmpfs, that is, the virtual memory (VM) file system we know. Tmpfs may be the best RAM disk-like system available in Linux and a new feature of the 2.4 kernel. Next, we will briefly introduce another new 2.4 kernel function, called "bind installation", which brings great flexibility when installing (and reinstalling) the file system. In the next article, we will focus on devfs. Later, we will take some time to familiarize ourselves with the new ext3 file system.

Introduction to tmpfs
If I have to make it clear at once, I will say that tmpfs is like a virtual disk (ramdisk), but not the same. Like a virtual disk, tmpfs can use your RAM, but it can also use your swap partition for storage. In addition, a traditional virtual disk is a block device and requires commands such as mkfs to actually use it. tmpfs is a file system, not a block device. You only need to install it, it can be used. All in all, this makes tmpfs the best Ram-based file system I have the chance to encounter.

Tmpfs and Vm
Let's take a look at some of the more interesting features of tmpfs. As I mentioned earlier, tmpfs can use both Ram and swap partitions. This may seem a bit arbitrary at the beginning, but remember that tmpfs is also known as the "virtual memory file system ". In addition, you may also know that the virtual memory resources in the Linux kernel are also derived from your RAM and swap partitions. The VM subsystem in the kernel allocates these resources to other parts of the system and is responsible for managing these resources in the background. Generally, the ram page is transparently moved to the swap partition or from the swap partition to the ram page.

The tmpfs file system needs the page of the VM subsystem to store files. Tmpfs does not know whether these pages are in SWAp partitions or ram; it is the work of the VM subsystem to make such a decision. What the tmpfs file system knows is that it is using some form of virtual memory.

Not a block Device
This is another interesting feature of the tmpfs file system. Unlike most "standard" file systems, such as ext3, ext2, XFS, JFS, reiserfs, and other systems, tmpfs does not exist on an underlying block device. Because tmpfs is directly built on the VM, you can use a simple mount command to create the tmpfs file system.

# Mount tmpfs/mnt/tmpfs-T tmpfs

After executing this command, a new tmpfs file system is installed in/mnt/tmpfs and can be used at any time. Note that you do not need to run mkfs. tmpfs; in fact, it is impossible because there is no such command. After the mount command is executed, the file system is installed immediately and can be used. The type is tmpfs. This is quite different from how to use Linux Virtual Disks. standard Linux Virtual Disks are Block devices, so you must format them with the file system you selected before using them. On the contrary, tmpfs is a file system. Therefore, you can simply install it.

Advantages of tmpfs

Dynamic file system size
You may want to know the size of the tmpfs file system we have installed in/mnt/tmpfs. The answer to this question is a bit surprising, especially when compared with a disk-based file system. /Mnt/tmpfs will initially have only a small space, but with the copy and creation of files, the tmpfs file system driver will allocate more VMS, and dynamically increase the space of the file system as needed. In addition, when files in/mnt/tmpfs are deleted, the tmpfs File System Driver Dynamically reduces the file system and releases VM resources, in this way, the VM can be returned to the loop for other parts of the system to be used as needed. Because VM is a valuable resource, you do not want to waste anything more than what it actually needs. One of the advantages of tmpfs is that these are all automatically processed. See references.

Speed
Another major benefit of tmpfs is its lightning speed. Because the typical tmpfs file system will completely reside in Ram, reading and writing can be almost instantaneous. Some swap partitions are used, and the performance is still excellent. When more idle VM resources can be used, these tmpfs file systems will be moved to ram. It is actually good for the VM subsystem to automatically move some tmpfs file systems to swap partitions, because doing so can free up space for processes that require Ram. This, along with its ability to dynamically resize, provides much better overall performance and flexibility for the operating system than traditional RAM disks.

No persistence
This may not seem like a positive factor. tmpfs data will not be retained after the restart, because the virtual memory is essentially easy to lose. I think you may have guessed one of the reasons why tmpfs is called "tmpfs", isn't it? However, this is actually a good thing. It makes tmpfs an excellent file system that stores data that you do not need to retain (such as temporary files, which can be found in/tmp and some parts of the/var file system tree.

Use tmpfs
To use tmpfs, You need to enable the 2.4 series kernel of the "virtual memory file system support (previously shm fs)" option; this option is in the "file systems" section of the Kernel configuration option. Once you have a kernel that enables tmpfs, you can start to install the tmpfs file system. In fact, it is a good idea to enable the tmpfs option in all your 2.4 kernels, whether or not you plan to use tmpfs. This is because you need kernel tmpfs support to use POSIX shared memory. However, System V shared memory does not need tmpfs in the kernel. Note that you do not need to install the tmpfs File System for POSIX shared memory to work; you only need to support tmpfs in the kernel. POSIX shared memory is currently not used much, but this situation may change over time.

Avoid low VM
Tmpfs dynamically increases or decreases as needed: If your tmpfs file system increases to the extent that it consumes all the virtual memory, and you do not have any remaining Ram or swap partitions, what will happen at this time? In general, this situation is a little annoying. If it is the 2.4.4 kernel, the kernel will be locked immediately. For the 2.4.6 kernel, the VM subsystem has been corrected in many ways. Although it is not a good experience to exhaust the VM, it will not completely fail. If the 2.4.6 kernel is unable to allocate more VMS, you obviously do not want to write any new data to the tmpfs file system. In addition, some other things may happen. First, other processes in the system will not be able to allocate more memory. Normally, this means that most of the systems will become extremely slow and have almost no response. In this way, it is very difficult or time-consuming for super users to take necessary steps to mitigate such a low VM.

In addition, the kernel has a built-in final defense system to release the memory when there is no available memory. It will find the process that occupies VM resources and terminate the process. Unfortunately, this "terminate process" solution usually causes adverse consequences when the usage of tmpfs increases and the VM is exhausted. The following are the reasons. Tmpfs itself cannot (or should not) be terminated because it is part of the kernel rather than a user process, there is also no easy way for the kernel to find out that the process is full of the tmpfs file system. Therefore, the kernel will mistakenly attack the process that can be found to occupy the largest Vm, usually X server, if you happen to be using it. Therefore, your X server will be terminated, but the root cause of the low VM situation (tmpfs) is not resolved. Ick.

Low VM: Solution
Fortunately, tmpfs allows you to specify the maximum file system capacity when installing or re-installing the file system. In fact, from the 2.4.6 kernel to the 2.11g kernel, these parameters can only be set during installation, rather than re-installation, but we can expect to be set during re-installation in the near future. The optimal setting for the maximum tmpfs capacity depends on resources and the usage mode of your specific Linux host; this idea is to prevent a fully-used tmpfs file system from consuming all the virtual memory results from causing the bad low VM situation we mentioned earlier. A good way to find a good tmpfs upper limit is to use top to monitor the usage of swap partitions in your system during peak usage. Then, make sure that the specified tmpfs upper limit is slightly less than the total number of idle swap partitions and idle Ram during all these peak usage periods.

It is easy to create a tmpfs file system with the maximum capacity. To create a new tmpfs file system with a maximum size of 32 MB, type:
# Mount tmpfs/dev/SHM-T tmpfs-O size = 32 m

This time, we did not install the tmpfs File System in/mnt/tmpfs, but created in/dev/SHM, which is exactly the "formal" Installation Point of the tmpfs file system. If you are using devfs, you will find that this directory has been created for you.

Also, if we want to limit the file system capacity to 512 KB or less than 1 GB, we can specify size = 512 K and size = 1g respectively. In addition to capacity limit, we can also limit the index node (File System Object) by specifying nr_inodes = x ). When nr_inodes is used, X can be a simple integer, followed by a K, M, or G to specify thousands, millions, or billions (!) Index nodes.

In addition, if you want to add the equivalent function of the above Mount tmpfs command to/etc/fstab, it should be like this:

Tmpfs/dev/SHM tmpfs size = 32 M 0 0

Install on existing installation points
Before using 2.2, an error occurs when you try to install anything again at the installation point that has already been installed. However, the overwritten kernel installation code makes multiple installation points no longer a problem. Here is an example: Suppose we have an existing file system installed in/tmp. However, we decided to start using tmpfs for/tmp storage. In the past, your only choice was to uninstall/tmp and reinstall your new tmpfs/tmp File System in its location, as shown below:

# Umount/tmp

# Mount tmpfs/tmp-T tmpfs-O size = 64 m

However, this solution may not work for you. Many running processes may open files in/tmp. If so, you will encounter the following error when trying to uninstall/tmp:

Umount:/tmp: device is busy

However, with the latest 2.4 kernel, you can install your new/tmp file system without the "device is busy" error:

# Mount tmpfs/tmp-T tmpfs-O size = 64 m

Run a command to install your new tmpfs/tmp File System on/tmp and on a partition that has been installed and cannot be accessed directly. However, although you cannot access the original/tmp, any process that opens the file on the original file system can continue to access them. In addition, if You unmount/tmp Based on tmpfs, the original/tmp file system will reappear. In fact, you can install any number of file systems on the same installation point, the installation point is like a stack; uninstall the current file system, the last recently installed file system will reappear.

Binding and Installation
By using binding and installation, we can install all or even some installed file systems in another location, and access the file system at the same time at two installation points. For example, you can use bind to install your existing root file system to/home/drobbins/nifty, as shown below:

# Mount -- bind // home/drobbins/nifty

Now, if you observe the inner of/home/drobbins/nifty, you will see your root file system (/home/drobbins/nifty/etc,/home/drobbins/nifty/OPT, etc ). If you modify the file in the root file system, you can also see the changes in/home/drobbins/nifty. This is because they are the same file system. The kernel simply maps the file system to two different installation points. Note: When you install a file system in another place, any file system installed at the point of installation bound to the installation file system will not be moved. In other words, if you have/usr on a separate file system, the binding and installation previously executed will leave/home/drobbins/nifty/usr empty. You will need to bind the installation command so that you can view the/usr content at/home/drobbins/nifty/USR:

# Mount -- bind/usr/home/drobbins/nifty/usr

Bind and install some file systems
Binding and installation makes even better things possible. Assuming that you have a tmpfs file system installed in its traditional location/dev/SHM, you decide to start using tmpfs in/tmp, which is currently in the root file system. Although you can install a new tmpfs File System in/tmp (this is possible), you can also decide to make the new/tmp share the currently installed/dev/SHM file system. However, although you can bind/dev/SHM to/tmp, your/dev/SHM contains directories that you do not want to appear in/tmp. So what do you do? In this way:

# Mkdir/dev/SHM/tmp

# Chmod 1777/dev/SHM/tmp

# Mount -- bind/dev/SHM/tmp

In this example, we first create a/dev/SHM/tmp directory, then grant it 1777 permissions and grant appropriate permissions to/tmp. Now that our directory is ready, we can install it or only install/dev/SHM/tmp to/tmp. Therefore, although/tmp/Foo maps to/dev/SHM/tmp/Foo, you cannot access the/dev/SHM/bar file from/tmp.

As you can see, binding and installation are very powerful, allowing you to easily modify the file system design without having to worry about it. In the next article, we will talk about devfs. Now, you may want to see the following references.

References

* Read previous articles in the Daniel series, where he introduced the benefits of creating logs and using reiserfs, and showed how to install a stable Linux 2.4-based reiserfs system.
* Linux weekly news is a good reference for keeping the latest kernel development in sync.
* Util-Linux (latest link) collects various important Linux applications, including Mount and unmount. You may want to upgrade to the latest available version so that you can use the Mount -- bind syntax instead of the Mount-O bind syntax ).
* Because tmpfs and the binding and installation are relatively new, most of them are new kernel features that are not described in the document. The best way to learn about these features is to learn about the Linux kernel source code.
* The namesys page is used to learn more about reiserfs.
* The reiserfs email list is a good resource for you to learn more about the current reiserfs information. Be sure to take a look at the reiserfs Mail List archiving.
* In the review of Juan I. Santos florido's Linux Gazette Journal File Systems, you can find in-depth explanations of the metadata differences between UFS, ext2, and reiserfs and other content.
* Jedi's reiserfs/Qmail optimization page contains many useful information for Qmail users. Check reisersmtp. Jedi collects many Qmail components that provide powerful Qmail performance.
* Read Steve best's JFS overview on developerworks.
* Join Daniel's free basic JFS tutorial on developerworks.
* Visit more Linux references on developerworks.
* Visit more open source code resources on developerworks.

About the author
Authordaniel Robbins is Gentoo Technologies, Inc. the President/CEO of Albuquerque, who lives in New Mexico, is the creator of Gentoo Linux (Advanced Linux on a PC) and portage systems (next-generation Linux porting systems. He is also the author of MacMillan's books Caldera OpenLinux unleashed, SuSE Linux unleashed, and Samba unleashed. Daniel was dragged into the computer from his second-year exposure to the potentially dangerous magic of Logo programming language and Pac Man games. This may explain why he used to be the chief graphic designer of Sony Electronics publishing/gaming companies. Daniel likes to spend time with his wife Mary and his newborn baby daughter hadassia. You can useDrobbins@gentoo.orgContact him.