SQL Removal Duplicate Delete duplicate data (Tens data volume)

MySQL has 5 million data, but most of it is repetitive, true on the 1.8 million, so want to how to get these duplicate data out, find a lap on the internet, a lot of it is not in such a code, so inefficient, their own pondering a combination of, find an efficient way to deal with this way, 5 million of data , 10 minutes to remove all repeat, please refer to.

Step One: Extract the data_content_152 fields from the 5 million datasheet SFZHM the corresponding ID fields to the TMP3 table

The code is as follows	Copy Code
CREATE TABLE Tmp3 as select min (id) as col1 from data_content_152 Group by SFZHM;

Step Two: Create a new table res

CREATE TABLE ' Res ' (

The code is as follows	Copy Code
' ID ' int (11), ' Sfz ' char (20) ) Engine=myisam;

Step three: Add the TMP3 table ID to the data_content_152 data to be extracted in the SFZ field of the Res table
INSERT into Res (SFZ) SELECT sfzhm from Data_content_152,tmp3 where data_content_152.id=tmp3.col1
At this point, it is implemented in MySQL, to the data table data_content_152 completely delete duplicate data, the data to be repeated to the Res table.