Suzhou freenas+esxi5 Data Recovery case

Unit

A network company in Suzhou

"Physical and logical storage"

The company uses an inexpensive storage model that uses iSCSI to achieve FC San functionality.

The physical storage architecture is on a DELL server, using FreeNAS for ISCSI, and then using two Dell servers to do the ESXi5.0 virtualization system.

FreeNAS layer for the UFS2 file system, the entire storage to build a sparse mode of file, hanging to ESXi5.0 system.

There are 5 virtual machines running within the ESXI system, three of which are the most important.

A windows2003 system virtual machine is the local portal of this company. Using the ASP. NET and PHP hybrid architecture, use the database for SqlServer2005 and MySQL 5.1.

One for the FreeBSD system, running MySQL database for use by many other virtual machines.

One for the windows2003 server, which stores the newly developed program code for this company.

"Failure Phenomenon"

After a sudden outage of storage, ESXI system is not stored, the administrator found the UFS2 file system problem in FreeNAS, and then the administrator with fsck repaired the file system, the ESXi system can be connected to the storage, However, it was found that the ESXi system failed to recognize the original data store and Vmfs file system, and the administrator formatted the VMFS and found nothing inside.

"Data recovery process"

The customer found several data recovery companies that took one weeks and still had no results. Things are too complicated.

After the introduction of a Shanghai operation and maintenance company, the customer contacted the Beijing North Asia Data Recovery Center.

After detailed communication, North Asia Data Recovery Center developed a detailed data recovery program, after customer approval, North Asia Data Recovery Center engineers directly to Suzhou for data recovery.

Analyze failures to maximize the use of available information.

Start Cobwebs:

Application Architecture Hierarchy: FreeNAS (UFS2 file system –> A large sparse schema file) –> ESXi 5.0 (VMFS file system layer)-a virtual disk for a single virtual machine (Windows-ntfs file System/ FREEBSD-UFS2 file system).

The first step is to mirror the FreeNAS layer and then analyze the entire storage, discovering a large file with more than 900 GB, file name: Iscsidata.

Through the binary structure of the UFS2 file system, locate the inode data of the Iscsidata file, and find that the file has been rebuilt, and the inode pointer is pointing to a small amount of data.

The FreeNAS layer cannot be resolved and cannot go into the next VMFS layer analysis.

Collect the important structure of the UFS2 file system:

Block Size: 16KB

Segment Size: 2KB

Cylinder Group size: 188176 KB

UFS2 A data pointer is 8 bytes, and a block can store 2048 data pointers.

Then a level two pointer block can be stored: 2048*2048*16kb= 64GB data.

A three-level pointer block can store 64gb*2048= 128TB data

If you can find a level three pointer block to the Iscsidata file, you can solve the FreeNAS layer problem.

However, the Iscsidata file has been rebuilt, and the process and size are the same as the original, and some of the pointer blocks are estimated to have been overwritten.

The inode of the original Iscsidata file and the iscsidata of the newly created file are in one place, trying to search and no other useful inode appears.

Have to write the program to collect useful pointer blocks:

650) this.width=650; "class=" Size-full wp-image-2055 aligncenter "alt=" image002 "src=" http://blog.datahf.net/ Wp-content/uploads/2013/07/image002.jpg "width=" 554 "height=" 444 "style=" border:1px solid rgb (233,237,241); height: Auto;padding:4px;margin:1em auto; "/>

Since the Iscsidata file is using sparse mode, the collection condition can only be relaxed, collecting a large number of three-level pointer blocks and two-level pointer blocks. The analysis of all the three-level pointer blocks collected is not valid, the three-level pointer block used by the Iscsidata file is estimated to be overwritten when new Iscsidata file is created (the new Iscsidata file has a VMFS formatting process after mounting to ESXi5.0, and ESXi5.0 uses GPT partitioning, the GPT partition writes the redundant GPT header and partition table information data at the end of the disk, which uses a level three pointer block of the Iscsidata file.

You can now analyze only the collected level two pointer blocks, dump the pointer data with a large number of two-level pointer blocks, and then position the data from the disk to a level two pointer.

This gets a lot of dump data.

Start analyzing the VMFS layer:

Reformatting the Vmfs, and the original UFS2 pointer has been lost, resulting in the Vmfs meta-file is basically unavailable, no important reference information, fortunately, the virtual machine has no snapshot, can still recover.

Through a single virtual machine layer (Windows (NTFS) and FreeBSD (UFS2) System file system structure), up to the VMFS layer, in the VMFS layer to locate the dump out of a single 64GB file, through multiple combinations, Finally, the virtual disks of all three important virtual machines have been fully recovered.

The customer uploads the recovered page data and database data to a newly built system, pulls up the application, and the data is completely free of problems.

650) this.width=650; "class=" Alignnone size-full wp-image-2056 "alt=" image004 "src=" http://blog.datahf.net/ Wp-content/uploads/2013/07/image004.jpg "width=" 554 "height=" 259 "style=" border:1px solid rgb (233,237,241); height: Auto;padding:4px;clear:both;margin:1em 0px; "/>

"Data Recovery Results"

It took 4 days and the final data 100% recovered successfully.

This is also to maintain the North Asia Data Recovery Center Yin engineer out-door service 100% success rate, there is still no case of failure.

"Data Recovery Engineer"

North Asia Data Recovery Center – Gongli

4006505646

Posted in file system, Server Tagged esx Recovery, ESX data recovery, VMFS recovery, VMFS data recovery, virtual machine recovery

Suzhou freenas+esxi5 Data Recovery case