MPQ Storm Library Source code Analysis a

MPQ what?

MPQ wiki the words are very clear.

In short, it's a tool that Blizzard uses to package game data. StarCraft, Warcraft games are used. This tool contains game resources such as encryption and compression functions.
git:https://github.com/stormlib/StormLib

The MPQ file has a total of four formats in common:

The first type: FLAT format . Linear storage.

Another type: Partial format , sub-package storage.

The third type: MPQE format , encrypted storage.

Fourth type: BLOCK format . Module storage.

The FLAT format is the simplest. Only this format also supports the encryption and compression of individual files.

This paper mainly analyzes the flat format , including the following five points:

First, create a MPQ file.

Two. Join the resource file.

Three, package MPQ file.

Four, read the MPQ file.

v. Read the resource file.

figure I, figure II, figure three is the internal format of the FLAT format .

Figure A

a , create a MPQ file.

the main process is. call the Sfilecreatearchive interface to create the MPQ file. After a successful creation, the interface returns success or not. At the same time, the handle of the MPQ file is assigned to the variable passed in.

The specific process for creating a MPQ file is:

1 "Initialize an encryption block in memory Stormbuffer;

2 tries to open a MPQ file with the same name. If a file that already exists with the same name is not a MPQ file, the MPQ tool tries to read the file in the format of the MPQ file. To avoid errors with the same name. In the stormlib example, you first delete the same file with the same name as you want, and then create the logic for the MPQ file.

3 " Create an empty MPQ file.

4 "Resets the file size.

5 writes a false MPQ file header, which is the area where a of figure one points.

MPQ's head has four different forms. Each has a different length, but the default is a maximum length of 208 bytes. Write this fake MPQ file header, the most basic purpose is to occupy a bit, to ensure that the first data file is written in the correct location.

6 "If it is a large file, then create the index table hettable of the large file.

7 "Create HASHTABL, which is used to index files.

The above seven processes. The main implementation of the two results, 1, the creation of the MPQ file, 2, in memory to generate the required MPQ data.

Second, join the resource file.

The main process is to call the Sfilecreatefile interface. Create a new file handle in the MPQ file. Writes data through the Sfilewritefile interface.

Call Sfileclosefile to free memory resources.

Specific process:

1 "Check if the internal file is not. MPQ mainly has two internal files, one is listfile. The other one attributes. If it is not an internal file, add a new file to the MPQ file.

2 "Find the location that can be written in the MPQ file. Assume that it is the first time you write. This is the area where figure one B is pointing.

However, there is no write at this time.

3 "Through the HashString method, the resource file name is converted to a hash index value. Take a TMPQHASH data structure from the current hashtable with the new hash index value .

4 If the new document is not created for the first time. At this point, The Dwblockindex value is passed Tmpqhash. Do dwblockindex offset in filetable. Gets the entry tfileentry for the new file. If it is the first time to create a new file. You will first get a spare position in the filetable and assign a tfileentry. The default is to take the filetable position for the dwfiletablesize offset, and at this point the Dwfiletablesize value is 0, so it is the first position in the Filetable table. That is , the area to which figure a B points .

Assign a tmpqhash at the same time (the value of the hash index is related to the name of the file), after the allocation is complete. associated with the fileentry. In Fileentry, the position of the Dwhashindex index into the hash table is tmpqhash by Dwblockindex Index to Fileentry.

5 "After you get the Fileentry, you initialize the other values of fileentry in the Sfileaddfile_init function.

Through these logic. will be able to get a tmpqfile file pointer to write the data.

6 "Write the file data through the Sfilewritefile interface, can write the resource file in batch to the MPQ file." It is also possible to write resource files to the MPQ file one at a time, both of which are not problematic.

Figure II

7 "Before the file is to be written, a sector is written, with a default of 8 bytes, and the sector is associated with the encryption of the data that will be written. Then write

Data , data has options, ability to choose whether to encrypt, compress, select data stored as a unit file.

After the write is complete, it is written again when

The front sector, detailing why the sector was written two times, and the next article continues the analysis. In Figure two. L point to the real resource file data, K, M are each fan

Area. Resource files can be selected for encryption and compression.

Might

Third, encapsulation MPQ file.
Encapsulation is important, and the package will update the data associated with the newly generated file to the MPQ file header. Cover the original false head.

The main process of encapsulation is. will be associated with the file Het table and the Bet table, as well as the hash table, file filetable. Writes sequentially to the end of the last resource file.

After the encapsulation logic has run out. The new MPQ file internal structure three.
The detailed process is:
1 "Determine if you want to write listfile, attributes and other files, if you want, it will be created in the MPQ file, and write.
2 "Call sfileclosearchive, find the position that can write Het, BET, HASH, filetable table, and then write the data of these tables into the new MPQ file. After writing the MPQ file inside three. The data for the table is written after the nth data file. Finally, go back to the head to update the MPQ file.

3 frees the resources in memory.

Four, read the MPQ file.

The main process is. Open the MPQ file and read the header first. After reading correctly, parse the location of each table. Gets the data file location. and a random link error. Will exit.
Detailed process:
1 " Initializes a Stormbuff (related to data encryption). Then , with a length of 208 bytes, read from the head of the MPQ file at 0 bits, reading 208 bytes at a time.

Forced conversion to Tmpqheader data structure after reading.

If the data inside the Tmpqheader is correct, the MPQ will have a lot of tests in order to infer if the file is correct. Only the MPQ file header has four formats by default.

Each format has a different length. After the head is determined. The memset is called. Place the excess part 0 to ensure that the Mpqheader data is correct.

2 through the newly acquired head data structure. Read the hash table. The hash table passes the Dwhashtablepos property of the Tmpqheader. Find its location in the file and read it out. The data in the file is then read into the memory, and the hash table data is read.

3. Create a file data sheet. Initialize the file data table filetable by reading the hash table. With the filetable can get fileentry, with the file entrance. will be able to read the data .

4 "Loading listfile and attributes files.

I'm not going to use these two options right now.

5 "returns the MPQ file handle.

v. Read the resource file.
There are two ways to read.

The first is to take out one of the specific files, and the other is to traverse all the data files inside the MPQ file.
The first way to take out a file is simpler.
The detailed process is:
1 "Call the Sfileopenfileex interface (ex is the meaning of the external file). The file name is passed to the interface, and the file name is converted to hash data inside the interface. By the Dwblockindex property of hash data, the fileentry data is calculated, and with Fileentry, the location of the data of the file to be read is obtained. And gets the handle to the data file. It is also possible to call Sfilehasfile first to infer whether the file is available.

2 "Call Sfilegetfilesize to get the file size, request memory space. Call Sfilegetfileinfo to get the file information, not the data.
3 "Call Sfilereadfile to get the data for a specific resource file in the MPQ file.

4 "Call Sfileclosefile to release the requested memory.

The other one is slightly more complicated.
the detailed process is:
1 "Call Sfilefindfirstfile, the match is *, then all matches. Returns a Hfind value.

2 Running the process of reading a file, as explained above, is not mentioned here.
3 "Call Sfilefindnextfile to get the next file entry.

4 "cycle run 1==>3 process. Until there are no files to read.

Copyright notice: This article blog original articles, blogs, without consent, may not be reproduced.

MPQ Storm Library Source code Analysis a