Build a yaffs2 File System on NAND Flash

Create a yaffs2 File System on NAND Flash (1)

By
On
2006-07-22

After more than half a month of hard work, we finally completed nandflash's MTD driver and the upper-layer yaffs2 file system. In the past half a month or so, we have to fight against frustration almost every day. Every day, we have to endure this bug. I think I have been reading a lot of materials over the past half a month. I have gained some help. In view of the fact that there is still little information on the domestic yaffs2 file system, I will share my experiences with you.
Do not gossip, first introduce some background information 1. Flash Memory
We often say that flash memory is just a general term. To be precise, it is commonly known as non-random access memory (NVRAM). It features that data does not disappear after power failure, therefore, it can be used as external memory. The so-called memory is a volatile memory, which can be divided into two categories: DRAM and SRAM. The memory is usually referred to as dram, that is, the familiar DDR, DDR2, SDR, Edo, and so on. Flash memory also has different types, which are mainly divided into two categories: nor type and NAND type.
Classification of flash memory
The nor flash memory differs greatly from the nand flash memory. For example, the nor flash memory is more like memory and has an independent address line and data line, but the price is relatively expensive and the capacity is relatively small; the NAND type is more like a hard disk, and the address line and data line share the I/O line. Similar to the hard disk, all information is transmitted through a hard disk line. In addition, compared with the nor flash memory, the NAND type, the cost is lower, while the capacity is much larger. Therefore, nor flash memory is suitable for frequent random read/write scenarios. It is usually used to store program code and run directly in flash memory. Mobile phones are large users who use nor flash memory, therefore, the "Memory" Capacity of mobile phones is usually not large. nand flash memory is mainly used to store data. Our commonly used FLASH products, such as flash drives and digital storage cards, use nand flash memory.
Here we also need to correct the concept that the flash memory speed is actually very limited, and its operation speed and frequency are much lower than the memory, in addition, the operating efficiency of nand flash memory is much slower than that of memory direct access. Therefore, do not think that the performance bottleneck of the flash drive is the interface, or even take it for granted that the flash drive will achieve a huge performance improvement after using the USB interface.
As mentioned above, the operation efficiency of nand flash memory is low, which is related to its architecture design and interface design, it is indeed like a hard disk (in fact, the compatibility with the hard disk is indeed considered at the beginning of the design of the nand flash memory), its performance is also similar to the hard disk: the operation of small data blocks is very slow, the speed of big data blocks is very fast. This difference is far greater than that of other storage media. This performance feature deserves our attention.
Technical Features of nand flash memory
The basic storage unit of memory and nor flash memory is bit. Users can randomly access any bit information. The basic storage unit of the nand flash memory is page (as you can see, the page of the nand flash memory is similar to the hard disk sector, and one hard disk sector is also 512 bytes ). The effective capacity of each page is a multiple of 512 bytes. The so-called effective capacity refers to the part used for data storage. In fact, we need to add 16 bytes of verification information. Therefore, we can see in the flash manufacturers' technical materials "(512 + 16) byte. At present, the vast majority of nand flash memory with less than 2 GB capacity is (512 + 16) bytes of page capacity. The nand flash memory with more than 2 GB capacity will expand the page capacity to (2048 + 64) bytes.
The nand flash memory is erased in blocks. The Flash write operation must be performed in a blank area. If data already exists in the target area, it must be erased before writing. Therefore, the erasure operation is the basic operation of flash memory. Generally, each block contains 32 512-byte pages with a capacity of 16 KB. When a large-capacity FLASH memory uses 2 kb pages, each block contains 64 pages with a capacity of kb.
Each nand flash memory has eight I/O interfaces. Each data line transmits (512 + 16) bit information each time, and 8 is (512 + 16) x 8 bit, that is, 512 bytes. However, the large-capacity nand flash memory is also increasingly adopting 16 I/O Lines. For example, the Samsung k9k1g16u0a chip is a 64 m x 16 bit NAND FLASH memory with a capacity of 1 GB, the basic data unit is (256 +
* 16 bit, or 512 bytes.
During addressing, the nand flash memory transmits 8-bit address information packets through eight I/O interface data lines. Because the size of the flash memory chip is relatively large, a group of eight-bit addresses can only address 256 pages, which is obviously not enough. Therefore, an address transfer usually requires several groups to occupy several clock cycles. The address information of the NAND includes the column address (the start Operation address in the page), block address, and corresponding page address. They are grouped at least three times during transmission and take up three periods. As the capacity increases, there will be more address information and more clock transmission is required. Therefore, an important feature of nand flash memory is that the larger the capacity, the longer the addressing time. Moreover, because the transfer address period is longer than that of other storage media, nand flash memory is less suitable for a large number of small-capacity read/write requests than other storage media.

Ii. MTD
MTD is short for memory technology device. MTD supports memory-like memory, which serves as a bridge between underlying hardware and upper-layer software. At the underlying layer, it provides good driver support for either the nor type or nandflash. For the upper layer, it abstracts the interface functions required by the file system. In addition, because of the special characteristics of Flash (similar to block devices and character devices), MTD can use flash as both Block devices and character devices. With MTD, writing flash drivers becomes very easy, because the upper-layer architecture is ready. We only need to look at the flash Datasheet and write the bottom-layer control sequence.
Translated from MTD official website http://www.linux-mtd.infradead.org/archive/index.html
MTD is committed to designing a general subsystem in Linux for memory, especially flash.
The purpose of designing this system is that through the interfaces between the hardware driver provided by this system and the upper-layer system, we can easily write the driver for the new hardware.
For underlying hardware drivers, they provide read, write, and erase processes. The storage format of files is irrelevant to them (such as FTL and ffs2). The upper-layer systems are concerned with storing user data in an appropriate way.
MTD User Module
MTD provides users with five modules that can be directly used in the user space.
Character Device
Block Device
Flash Conversion Layer
Nandflash Conversion Layer
Jffs2 File System

Iii. yaffs2 File System
There are many file systems for flash. I know jffs (1, 2, 3) and yaffs (1, 2 ). There is also a commercial RFS (robust File System) developed by Samsung, which is a one-stop service dedicated to Samsung's own NAND and onenand, from the underlying driver to the upper-layer file system, and is known to be 100% compatible with the fat format. At that time, I saw my drool water and turned my teeth from Samsung. The following describes the open-source yaffs file system.
The yaffs (yet another flash file system) file system is an embedded file system designed specifically for NAND Flash. Currently, there are two versions: yaffs and yaffs2. Generally, yaffs has good support for the following 512 bytes/page, and the larger page requires yaffs2, such as 2 k/page.
The yaffs file system is similar to the jffs/jffs2 file system. The difference is that the jffs1/2 file system was originally designed for nor flash applications, nor flash and NAND Flash are essentially quite different, so although jffs1/2 file systems can also be used in NAND flash, however, since it makes some trade-off for the nor feature in terms of memory usage and startup time, it is usually not the optimal solution for NAND.
Yaffs treats all the content (such as normal files, directories, links, and Device Files) on the file system as files. Each file has a page dedicated to storing the file header, the file header stores information such as the file mode, owner ID, group ID, length, file name, and parent Object ID. Because the content needs to be put down on one page, the length of the file name and the path name of the symbolic link object are limited.
As mentioned above, each page of data on NAND Flash has additional space to store additional information. Generally, the NAND driver only uses a part of the space, yaffs uses the rest of the space to store file system-related content. In addition, because the page size is increased, yaffs2 uses more spare spaces to store the information. In the structure of the yaffs and yaffs2 have a certain degree of difference, the specific structure can take a look at this document http://www.aleph1.co.uk/node/38
How does this file system work.
The first step of operating the file system is to obtain superblock. The yaffs file system itself does not have the so-called superblock block on NAND Flash, it is completely filled by the read_super function during the Mount process of the file system. However, it is interesting that the superblock is not stored physically, therefore, the yaffs File System on NAND Flash does not store the magicnum of filesystem, And the s_magic parameter in the superblock in the memory is directly assigned a value, therefore, any file system stored in NAND flash can be mounted as a yaffs file system, but the data will be stored as error data in the lost + found directory, I don't know if this is a bug in the yaffs file system.
Generally, a specific file system has its own dedicated data in addition to common data in the super_block structure of VFS. The dedicated data of the yaffs file system is a yaffs_devicestruct structure, it is mainly used to store related hardware and software configuration information, related function pointers, and statistical information.
When read_super is executed during the Mount process, the yaffs file system also needs to establish the directory structure of the file system in the memory. Because there is no super block, you need to scan the yaffs partition and determine whether the file header page or data page is based on the yaffs_tags Information read from OOB. Create a corresponding yaffs_object object for each file (object) in the memory based on the content of the file header page and the objectid, chunkid, and serial number in the data page.
In the yaffs_object structure, it mainly includes:
Such as modification time, user ID, group ID, and other file attributes;
Used for yaffs file system maintenance, such as dirty mark and delete mark;
Used as an organizational structure, such as pointing to the parent pointer of the parent directory, pointing to the siblings bidirectional linked list header structure of other object linked lists in the same level directory
In addition, according to different object types (directories, files, links), corresponding to a specific type of object, there are their own proprietary data content in yaffs_object
Common File: file size, used to quickly find the yaffs_tnode tree pointer of the file data block, etc.
Directory: Directory item content two-way linked list header (children)
Link: The objectid corresponding to the alias and hardlink of softlink
In addition to the yaffs_object created for the objects stored in NAND Flash, some virtual objects (fake object) will be created during the read_super execution ), these fake objects do not have corresponding physical entities on NAND Flash. For example, at the beginning of the file directory structure, yaffs will create four virtual directories (fake directory): rootdir, unwritable Dir, deletedir, lostnfounddir is used as the root directory, The unlinked object directory, the delete object directory, and the zero-point data block directory.
By creating these yaffs_object, The yaffs file system can organize the data systems stored on the NAND Flash and maintain a complete file system structure in the memory.
In addition, when I looked at the source code of yaffs2, I found that the mount of yaffs2 is different from that of umount and yaffs. I added a checkpoint mechanism. Every time I was at umount, yaffs2 will open up several blocks to access some information, so you don't need to scan the entire flash when you mount the next time, but you only need to find these blocks, which can greatly accelerate the Mount time. However, I have not studied the principle carefully. If you are interested, take a look at this mail http://www.aleph1.co.uk/pipermail/yaffs/2006q2/002019.html

Tired. come here first, and write and transplant next time.