U disk version of portable Linux production, CASPER-RW resolution __linux

I have been thinking that no matter whose computer I use, I can get a completely consistent working environment, with my favorite software, the important information I keep, and even the browser's preferences are identical. Today's cloud computing technology can partly solve the problem, but it's not nearly enough. My ideal state is, no matter where, a boot, see is their own computer, or equivalent to their own computer. They can handle their own data, do not disclose privacy to others, of course, do not destroy the existing software environment. To achieve this ideal, there are several ways:

1. Take a notebook or netbook at any time. Use it wherever you go.

2. With a small U disk, which installed the operating system and software, where all can use it.

3. As long as there is a network account, where you can get a dedicated to their own security software environment (including operating system and application software, as well as personal data).

Approach 1 too troublesome, option 3 needs the further development of cloud computing technology, and network bandwidth further upgrade. Method 2 The most realistic, also best use. This article focuses on how to install a Linux disk, your own software and data can be permanently stored on it.

A. Version Selection

Linux has a lot of versions, and here you choose Ubuntu. The reason is easy to use, the official and civil support is good, the PC or notebook compatibility is good, the content of its own integration has been able to meet the basic needs of desktop users, in the above with the Package Manager to obtain software is also extremely convenient. Not like other distributions, often to support a driver, or to install a particular software has to be repeatedly set up.

The latest stable version is Ubuntu10.04. Portable u disk is to provide a convenient office environment, so you should choose the desktop version, do not need to select the server version, unless hard to do a mobile server, hehe. Between the 32-bit and the 64-bit two versions I hesitated for a moment. Theoretically 64-bit performance is better (so the server is typically 64-bit), however, 32-bit compatibility is naturally more than 64 bits (many software only 32-bit version), and the U disk system may run on the older 32-bit CPU, so eventually I finally chose 32.

B. The production of Live USB

Many Linux support the production of live CDs, and Ubuntu is certainly no exception. The Live CD can not only guide the computer to install the system, it is itself a rich Linux system, pre-installed with the commonly used software, completely through the CD-ROM boot can go to the desktop, browse the Web, chat online, edit the file, everything is nothing. Even we can install the software, but since all of our changes are in memory, after restarting the computer, the new software is gone and all the changes have disappeared.

Now Ubuntu also supports the production of live USB, eliminating the trouble of burning discs. Almost all newer computers support USB booting, so Live USB can be easily used to boot the system. Of course, it has all the features of the live CD, and more importantly, the USB disk itself is readable, and we have even had the chance to use it to save our changes, which is much better than live CDs.

I downloaded the universal USB on the official website Guided tour (http://www.ubuntu.com/desktop/get-ubuntu/download) Installer (version 1.7.9.8), it is easy to use a 4G u disk on Windows to make a live USB. Use it to start the computer with interest, and see the beautiful Ubuntu desktop. So, I gave it a good network, installed a number of commonly used software. After a while, I turned off my computer and plugged the U-disk into another computer. After startup, the system becomes clean again, and its own changes are completely intact. This isn't exactly the same as a live CD.

Internet search for some, there are many production can permanently retain user data of the U disk version of Ubuntu method. For example, this article mentions several methods: Https://wiki.ubuntu.com/LiveUsbPendrivePersistent. The most direct is to direct the USB disk as a hard disk to install. This installation principle is simple, but the wear of U disk is also the biggest. We all know that the USB disk is used to store the chips are NAND Flash. Flash chip each block has a certain number of erase/write limit, more than this limit the block is likely to be broken. The typical number of writes is 10,000 times and 100,000 times. We usually use the large volume U disk is generally used MLC flash chip, write cycle is about 10,000 times. U disk in the firmware of the wear balance and bad block processing algorithms, we use it as the usual data storage media, more than enough, do not worry about writing 10,000 times will be a U disk write bad situation. But if used to run the system directly, because of temporary files and many other factors, its written pressure will be much more, in this case, we have to consider the use of U disk Time. Therefore, the direct use of U disk when the hard drive to install Linux is not recommended. Because of this, even if the swap partition is banned, the/TMP partition will also be on the hard disk, the partition's temporary files will speed up the wear of u disk. I http://rudd-o.com/en/linux-and-free-software/in a different way. A-better-way-to-create-a-customized-ubuntu-live-usb-drive to make a U disk, but after reboot, only parked in the Grub console, unable to start Linux. As I prepared to elaborate on this approach, I found that universal USB Installer originally supported the production of a USB drive that can persist data, but I didn't notice it before. Take a look at the picture below:

I ignored step 4 and used the default value of no persistence. In fact, the purpose of this option is to allow you to choose the right size to save your own modified data. I chose 2GB CASPER-RW so that any changes I make to the system (including new software, added files, etc.) will be present CASPER-RW the file, which can be 2G. Of course, users can not care about CASPER-RW this file, in accordance with the past use of the U disk Ubuntu.

Speaking of this, the creation of a U disk version of Ubuntu is over. The use of tools to do a U disk, can be almost in any mainstream computer to start. The installed software will also be preserved in accordance with the original, cool. If you're simply trying Linux on a USB drive, you can skip the rest of this article when you see it. If you want to know about some of the technical details of USB u disk Linux, want to use a U disk to effectively move the office, please continue to look down.

C. u disk partition

USB disk is of course the larger the volume the smaller the smaller. Now 4G and 8G of u disk are very cheap. I have the most of the U disk for 4G, so this article in the size of the U disk default is 4G. I do a u disk, can also be the entire U disk content back to a file inside, the system can be restored at any time, and can easily be cloned to another USB drive, so that my mobile work environment further practical-it does not need to physically rely on a specific USB drive, as long as I have a backup of the image file, You can restore the entire system at any time.

However, the actual capacity of the 4G U disk does not have 4G. This is mainly due to different unit conversions. In Windows, is in accordance with 1k=1024 Byte, 1m=1024k,1g=1024 to calculate, u disk manufacturers generally according to 1k=1000 Byte, 1m=1000k,1g=1000m to calculate. So, in Windows see the actual capacity of U disk is often 3.7-3.8g (different USB stick slightly difference). Suppose I have 2 4G USB drives, one with a capacity of 3.7G and one with 3.8G. If I install the system on the 3.7G u disk, and then copy it to the 3.8G u disk, there is no problem, but conversely, there will be a part of the data thrown away, which may lead to potential problems. To improve compatibility, I'm zoning the U disk and zoned a 3.5G FAT32 partition to make the system. Universal USB Installer making U disk Linux, it seems that you must require the U disk itself has been Gecheng fat format, or you can not select the disk. In fact, the U disk Linux partition itself is the FAT format, but the above file may be used as a block after Linux startup, in other file system format is loaded again (I will speak later). When you make your own Linux system, if you consider the convenience of replication and other compatibility factors, you can also reserve a little bit of capacity.

Windows Disk Management does not directly partition the USB disk, we can use the Third-party tools to do. I am using the binary Editing tool to open the U disk directly, modify the first sector (MBR) partition table to allow 3.5G size of the partition. The rest of the 200MB space, I'll keep it. Because Flash needs to use the wear balance algorithm to improve the service life, this 200MB additional space is used to the overall wear balance, to extend the use of U disk is likely to have some benefits.

D. aufs File System

Start the USB disk version of Ubuntu, enter the Mount command, get the following output:

aufs on/type aufs (rw)None On/proc type proc (rw,noexec,nosuid,nodev) None On/sys type Sysfs (rw,noexec,nosuid,nodev) None On/dev type Devtmpfs (rw,mode=0755) None on/dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)/dev/sdb1 on/cdrom type VFAT (rw,relatime,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1,shortname= Mixed,errors=remount-ro)/dev/loop0 on/rofs type SQUASHFS (ro,noatime)None On/sys/fs/fuse/connections type Fusectl (rw) None On/sys/kernel/debug type Debugfs (rw) None on/sys/kernel/security Type SECURITYFS (rw) None On/dev/shm type Tmpfs (rw,nosuid,nodev) Tmpfs on/tmp type Tmpfs (Rw,nosuid,nodev) None on/var/ru N type Tmpfs (rw,nosuid,mode=0755) None On/var/lock type Tmpfs (rw,noexec,nosuid,nodev) None ON/LIB/INIT/RW type TMPFS (rw , nosuid,mode=0755) Binfmt_misc on/proc/sys/fs/binfmt_misc type Binfmt_misc (Rw,noexec,nosuid,nodev) Gvfs-fuse-daemon ON/HOME/UBUNTU/.GVFS type Fuse.gvfs-fuse-daemon (Rw,nosuid,nodev,user=ubuntu)

We do not consider PROC,SYSFS, such as the system to assist the management of the special file system, only to consider the actual storage disk with the part of the deal. In the list above, we do not see the mainstream file system under Linux, such as Ext3/reiserfs, which is most familiar to us. That is what the USB disk set up on the file system, is how to save the data.

As mentioned earlier, when you make a U disk, you need a FAT32 partition on the U disk. The production tool copies the necessary files into the partition and does not make modifications to the partition itself. So, now the U disk is naturally having a FAT32 partition. From the mount output can be seen, the device/DEV/SDB1 (is the first partition of the U disk, the computer's local hard drive is recognized as/DEV/SDA) was mount to/cdrom, the format is VFAT. As the name suggests, the partition functions like a live CD, and we can simply understand that it contains the necessary content for an Ubuntu installation CD. Because all the software and data are stored on the/DEV/SDB1,/cdrom naturally contains all the content we need.

But from the/cdrom can find only a limited number of files, it seems that most of the installation-related, familiar with the/cdrom/casper directory of the kernel files Vmlinuz and Initrd.lz. It is not possible to find other Linux system files and configuration files that we are familiar with. Now that the/cdrom is all covered, where is the/bin,/sbin,/etc from?

/cdrom/casper There is a huge file filesystem.squashfs, about 700MB, from the name can guess, this file is not itself contains the necessary file system to run Linux. and its size can accommodate enough applications. In Linux, a single file can be loaded into a loop device, which can then be used as a common block device and mounted to other file systems. (file A can be used as a block device block_a, there is a file B, that file B can be used as a block device Block_b, there is a file C ... One layer, this is the "inception of the Space" file system version deduction. To view the current loop device information using the following command:

$ sudo losetup-a/dev/loop0: [0811]:27 (/CDROM/CASPER/FILESYSTEM.SQUASHFS)/DEV/LOOP1: [0811]:33 (/casper-rw-backing/ CASPER-RW)

The system creates two loop devices, where/dev/loop0 is created based on FILESYSTEM.SQUASHFS. Combined with the output from the previous Mount command, we were able to know that/dev/loop0 was mount to the/rofs directory and that the file system type was SQUASHFS. SQUASHFS is a read-only compressed file system, widely used on many Linux, fast, high compression ratio, is the first choice for Live CD production. In Ubuntu, a brief query,/rofs under a total of more than 100,000 files, the total capacity is about 1.7GB. It's good to think about the size of the original FILESYSTEM.SQUASHFS 700MB,SQUASHFS. You can also find out why you can put so much content on live CDs (live USB).

The/rofs directory has almost everything we need, including the familiar Bin,sbin,lib,etc,home directory. As if the entire Linux system had been incorporated into the bag. But it is only read-only (ROFs says read Only file system), we now USB U disk Linux on the filesystem is readable and writable. Moreover, there is a bin under the/rofs, there are/bin under the root directory, what is the relationship between the two. Compared,/rofs/bin and/bin seems exactly the same.

The secret lies in Aufs (Another Union FS), the special file system. It is not a common file system, but the existing different file systems together to form a unified file system. Suppose there are two file systems, Fs_a and Fs_b, and the directories they mount are/A and/b respectively. With Aufs, they can be merged, and/A and/b can be mapped back to the root directory/. In actual use, the general user only need to care about the root directory/the corresponding file system is Aufs can, do not care too much of the underlying details. When it comes to this, someone has a question. For example, to create a file under Aufs, whether the file exists under/A or below/b. There is no obvious conflict.

In fact, Aufs does not have the ability to aggregate arbitrary file systems together. It generally only combines two file systems, one of which is read-only and the other is readable and writable. Read from a read-only file system when the file is reading, or read from a writable file system if the file is available on a read-only filesystem. Writing a file can only be written to a writable file system. Read-only file systems can be considered to be the basis for a read-write file system that is supplemented by the use of copy on write (COW, write-time copying) to save user modifications. Because just save the modified part, so it can save a lot of space.

For the example of this article,/rofs is a read-only file system, and a file CASPER-RW is used to save the user's change data. CASPER-RW is a universal USB installer directly created in the root directory of U disk. If the size of the file is 2G, you can save 2G of modifications. If the disk space allows, we can add content to the end of the file to expand the file to increase the capacity of the data can be saved. The file itself is loaded into a loop device, and from the LOSETUP-A output, the device is/DEV/LOOP1. The ext2 file system is created on this device and is mount to the/cow. Aufs/rofs and/cow together, loaded into the root directory/below. /rofs put the original file,/cow save user changes. So we can see that the file structure under the root directory is basically the same as under/rofs, except that the new installed software and the newly created file cannot be viewed under/rofs. (Aufs's creation of the relevant script in Initrd, after startup, you cannot directly view directories such as/cow.) ）

We can easily understand that the USB disk version of the basic system of Ubuntu is stored in the file FILESYSTEM.SQUASHFS, its changes are stored in the file CASPER-RW.

E. Reduce the wear on U disk

Because of the flash chip erase/write times limit, we guarantee to meet their needs, under the premise of minimizing the number of U disk writes. By default, U disk version of Ubuntu has been banned from the swap partition, and the/TMP mount to the memory, these are extended U disk use time effective means.

In the system, the default time to access the file is recorded in the metadata for the file last accessed. The cost of this operation is actually very large, for example, I just open a file does not write any of the file, will cause the system to the bottom of the disk at least one write operation. For u disk, the update of access time is also an important factor in reducing performance and wearing a U disk. This time is of no practical use for ordinary applications, so we can put it off. (There are a few programs that rely on last access time to make some special judgments, but from the big picture, removing it is an important way to improve performance and reduce wear and tear on a USB disk version of Linux.) The Linux Mount command has a noatime option, plus it enables the ability to disable last access time. (There is also a disabled last access time option for the directory, called Nodiratime.) But some people analysis as long as adds the Noatime, will realize the Nodiratime function. For example, if you load the/DEV/SDC into a ext3 file system and ban the last access time, we can call mount:

Mount-t Ext3-o NOATIME/DEV/SDC/MNT/MYFS

In fact, in the process of making a U disk version of Ubuntu,/rofs has been set up noatime. Loads to/cow CASPER-RW and Aufs are also set Noatime (Mount output cannot be viewed directly, but can be seen in initrd.lz related scripts). In addition, the/cdrom is relatime to mount. Compared to Noatime, Relatime is a compromise that considers performance and functionality. It is updated only if the file's last access time is earlier than the final modification. Therefore, in a relatime way, you can record the time of the first visit after the file has been modified.

The log file system (such as EXT3) writes additional logs to the disk to ensure the integrity of the file system. But for u disk, the added log will undoubtedly lead to more write operations. Therefore, if it is not sensitive to the stability of the system, it is recommended that the Ext2 file system with no log be used on CAPSER-RW. In fact, the U disk version of Ubuntu in the initialization of the CAPSER-RW with the ext2.

In Linux, files are typically written to disk in asynchronous mode, except for special requirements. That is, the data that the application writes is first placed inside the page cache, and then the appropriate time to brush (flush) to disk. This approach can improve performance while also effectively reducing the number of writes. The Linux kernel derives several proc files that allow us to adjust the relevant flush parameters. USB disk version of Ubuntu has made some changes to the default parameters of Linux. The associated proc files and Ubuntu settings are as follows:

/proc/sys/vm/dirty_ratio:20/proc/sys/vm/dirty_background_ratio:10/proc/sys/vm/dirty_writeback_centisecs:1500/ proc/sys/vm/dirty_expire_centisecs:3000

The simple explanation is as follows: When the page cache's dirty data (dirty) exceeds the memory 20% (dirty_ratio), it is immediately brushed to disk, if memory 10% (Dirty_background_ratio) is exceeded, The process pdflush the data to disk asynchronously in the background. If the 10% threshold has not been reached, it does not matter, the kernel will start every 15 seconds (dirty_writeback_centisecs) flush operation, as long as the dirty data to stay in cache for more than 30 seconds (dirty_expire_ CENTISECS), will also be brushed to disk.

In order to further reduce the number of writes, so that the dirty data in the cache longer, we can change some of the parameters in the startup script, for example, modify the Dirty_writeback_centisecs to 6000, indicating the background of the periodic flush time to 60 seconds. The modified Dirty_expire_centisecs is 6000, which indicates that the dirty data timeout is 60 seconds. The other two parameters can not be modified, because in a small amount of writing, dirty data is generally not up to 10% of the total memory, in a large number of data writing, there is no need to cache the dirty data longer. Also note that after changing this data, the system suddenly loses power, and so on, the probability of data loss will be relatively greater. So how to choose, according to their own needs to weigh.

In addition, on the U disk, please only install the software you need, do not like ordinary Linux as a random installation. This is to reduce the number of U disk writing is very important and very simple a means.

F. Backup and restore

U disk version of Ubuntu made a number of settings, improve performance, the theory should also extend the use of U disk Time. But these operations also pose some risks, such as when the system suddenly loses power, the most recently modified data is more easily lost. We are running on this U disk is the desktop system, mainly for convenience, function enough, not as strict as the server requirements, but also to bear the risk (if there is a higher reliability requirements, please do not use this U disk System). However, even if the U disk quality is good, we also pay attention to it, it is always possible to damage, therefore, it is strongly recommended to regular backup of U disk. After the backup of the mirror file, can be restored at any time to the capacity of large enough USB disk, so that the system has a rapid cloning function. Even if you forget to bring a U disk, you can also get the mirror over the network (the network speed is fast enough), quickly restore a dedicated work environment.

Many of the tools under Windows have an entire disk copy capability. Linux can also be easily backed up with DD (assuming the device named/DEV/SDA for u disk):

DD If=/dev/sdb of=/mnt/backup/ubuntu_bak1.img bs=1m

A larger block size (BS) setting helps improve performance, especially when you write data to a U disk. (a chunk of writing helps reduce the bottom io of u disk.) For general Flash, 1 bytes at a time and one write 4K of Io are the same, or even more. If you insert another U-disk/DEV/SDC, you can copy the backup image to the U disk, so that the U disk immediately becomes my custom version of Ubuntu:

DD if=/mnt/backup/ubuntu_bak1.img OF=/DEV/SDC bs=1m

Of course, if you want to speed up the backup, you can also back up the CASPER-RW file separately, because the changes you made are saved in the file.