Talking about Exchange Server mail storage System-principle (2)

Log file We discuss the Exchange server's message store and have to talk about its log files. I've heard more than once about an Exchange Server administrator complaining that daily to file is soaring every day, consuming hard disk space. Let's see what these log files actually do. For each storage Group,exchange server generates a series of corresponding log files. The size of these log files is 5 m, with the extension log, and their prefix is e0x, where x is the number of the storage group corresponding to the log file [footnote: Although there is a text box in the storage group's properties "log file Prefix", But in fact, this cannot be changed. ]。 So the first storage group has a log file prefix of E00, and the second one is E01, and so on. The purpose of this is to prevent the administrator from "pigtailed" the log file during maintenance when there are multiple storage group. In addition to continuous log files, we can also see E0x.chk, Res1.log, Res2.log and other files. Many administrators have a headache with log files, so what is the purpose of Microsoft's introduction of log files in the Exchange Server database system? We look at the following aspects: 1. As an enterprise-class mail database system, data security and integrity must be foolproof. Must be able to face any possible crashes and outages at any time, what happens if we crash? To be able to reduce the loss of data to the latest level. 2. High-performance message throughput must be provided, and transaction operations on the messages in the database must be logged to the storage media immediately after completion (the persistence of the transactions). 3. When a disaster occurs, a backup recovery using the database must be returned to the database state the moment before the disaster occurred. Now let's take a closer look, when I want to modify the contents of the mailbox, the modified content is first read out and put into memory. The actual changes occur in memory, and when the modifications are complete, the content must be written back to the storage media to indicate that a modification was completed successfully. For such a modification process, at the database level, we are called a "transaction". We know that in order to ensure the integrity and consistency of the database, the operation of the transaction is atomic level. If a transaction succeeds, it signifies that the change he made has been permanently preserved; If a transaction fails, the system must return to the state before the transaction begins. When the system completes the modification in memory, the transaction is not completed. If this time goes down, the contents of the database are still unchanged. So, how do you make sure that the modifications made in memory can be written to the database at the first time?(To meet the requirements of database transaction persistence). Note that if the first time here, the faster the better. If we write directly to the EDB file, cannot do the fastest, because the EDB file is usually very large, the I/O system in the large file for random write operations, will spend a lot of time waiting for disk to find the appropriate track and sector, when the system is busy, this will be a bottleneck. Therefore, the database system uses log files, and when the changes in memory are complete, it is written to the log file first. Log files are small in size and write performance is much better than large EDB files. After the write is completed, the transaction is successfully saved on the storage medium. The database engine for Exchange Server writes the contents of the log file to the database in the background, because the transaction operation is completed, and the completed transaction is not lost even if the power is down or down at this time. This is the first function of the log file: To ensure that transactions are saved to non-volatile storage media at the first time. (Provide persistent durable support) as described above, we know that the Exchange Server database in operation is made up of three parts-the contents of the memory that have been modified but have not yet been written to the log file (Dirt Page). --There is no log file content written to the database file. --edb and STM files. For in-memory data (Dirt Page), this data is lost when the system loses power or crashes. Exchange Server uses a file named E0x.chk (Check Point) to record those log files that have been written to the database file. This is a pointer-like record.

We can use the command eseutil/mk to view the contents of this file chk

c:/.../exchsrvr/bin> eseutil/mk "C:/.../exchsrvr/mdbdata/e00.chk" Microsoft (R) Exchange Server (T M) Database Utilities Version 6.0 Copyright (C) Microsoft Corporation 1991-2000.  all rights Reserved. Initiating FILE DUMP mode ...     Checkpoint file:c:/program files/exchsrvr/mdbdata/e00.chk     LASTFU Llbackupcheckpoint: (0x0,0,0)     Checkpoint: (0x8,26da,30)     Fullbackup: (0x0,0,0)     Fu Llbackup time:00/00/1900 00:00:00     incbackup: (0x0,0,0)     Incbackup time:00/00/1900 00:00:00 &N Bsp   Signature:create time:03/28/2004 20:26:10 rand:6519986 computer:     ENV (CIRCLOG,SESSION,OPENTBL, verpage,cursors,logbufs,logfile,buffers)         (   off,    202,   10100,   1365,  10100,    128,  10240,  40828) operation completed successfully in 1.47 Seco Nds.

In the output of the command, Checkpoint: <0x8,26DA,30> represents the log full location currently committed to the database file. Where 0x8 is the ordinal of the log file, usually corresponds to E0x00008.log, and the remaining two parameters are the number of the log file internal pages (page). Let's look at the role of log files for system backup and recovery. As mentioned earlier, Exchange server requires that it be able to revert to the state of a disaster the moment before a disaster occurs. For a general system, we always make backups every week or every day, so how the data is protected after the backup and before the disaster occurs. The answer is a log file. We know that any changes to the database will be written to the log file before being updated to the database by the log file. We now assume that there is a system that backs up every day, and the system works after the backup is complete. If the system fails at 12:00 noon and the administrator restores the system with 3:00AM tape, the data from 3:00am to 12:00am will be filled by the log file. Specifically, when a backup of 3:00AM is completed, Exchange Server automatically scans the log folder associated with the store, and if a new log is found to exist than the current database, the Exchange The server automatically writes these logs sequentially to the database. As a result, changes made to the database from 3:00am to 12:00am can be restored back. This is a second important function of the log file. One might ask if the database files and log files are damaged at the same time, if the log file is not enabled. The answer is this: avoid this happening. First of all, the probability of database file corruption is much larger than the log file, in addition, Microsoft recommends that the database files and log files are placed on separate disks. We will focus on this issue in the next installment. The administrator complains about log files that these files grow daily and consume hard disk space a lot. The only reasonable solution to this problem is to make regular full or incremental backups of the storage group. Because Exchange Server deletes all log files that were generated before this backup after full or incremental backups are complete. It is not a good practice for many administrators to manually delete log files, or to start a "circular log" to reduce the consumption of hard disk space. Incomplete log files will cause the system to revert to the nearest state when the backup is restored. If your system does a full backup once a week, and you happen to have deleted some log files after the backup, then you may lose the data after the backup if you need to recover.Remember, data is always more valuable than disk space. startup and shutdown of the ESE database engine and the information Store serviceWhen the ESE engine loads a database file, it examines a special flag bit for the database file. This flag bit holds the last time the database file was closed gracefully. This state is represented by "consistent" or "inconsistent". For a normally closed database file, all the contents in the log file and in memory should have been submitted to the database file, only at this time, the database will be marked as "consistent". It is important to note that in a running database, its status must be "inconsistent" because the log file must have not been submitted to the database file content. For a database that has been closed and the status is marked as "inconsistent", it does not mean that the database library file is corrupted, and "inconsistent" simply means that the content that has not been written to the database file is saved in the log file. Use the command eseutil/mh to look at the shutdown state of the database.

c:/.../exchsrvr/bin> eseutil/mh "C:/.../exchsrvr/mdbdata/priv1.edb" Microsoft (R) Exc Hange Server (TM) Database Utilities Version 6.0 Copyright (C) Microsoft Corporation 1991-2000.  all rights Reserved. Initiating file DUMP mode ...      database:c:/program files/exchsrvr/mdbdata/priv1.edb file Type:databas E format ulmagic:0x89abcdef engine ulmagic:0x89abcdef format ulversion:0x620,9 engine ulversion:0x620,9 Created ulvers ion:0x620,9 DB signature:create time:03/28/2004 20:26:24 rand:6536656 computer:cbdbpage:4096 dbtime:63139 (0-63139) S Tate:clean Shutdown     <-----Indicates the status of database shutdown Log required:0-0 streaming file:yes Shadowed:yes last objid:57 4      ...< > ...     operation completed successfully in 1.391 seconds.

The "clean Shutdown" of the State field indicates that the database is in consistent status. When ESE loads a database file, it directly mount the store for the "consistent" database file, and for the "in consistent" database file, ESE will perform the process called "Soft Recovery", during which Log content that is not submitted to the database file in a timely manner is written to the database. When all logs are written, the database is marked as "consistent" and then loaded normally. At the beginning of Soft recovery, ESE writes the log file based on the location pointed to by the Check Point file (the database starts with the oldest log file if the check-point file is also corrupt or does not exist). When ESE writes data from the log file to the store, it determines whether the log file needs to be written to the database according to the dbtime timestamp.

Original source Http://bbs.5dmail.net (Chinese name: 5Dmail Mail Information Network)