Delete duplicate records in the database (delete only duplicate records)

Background:

Today, when I used a crawler to crawl data, I was accidentally suspended when I first crawled data. Later, I started again, but forgot to clear some junk data, as a result, many repeated records are generated in the wordlocation table.

The wordlocation table defines the following wordlocation (urlid, wordid, location), where (urlid, wordid) should be unique.

Requirements:

Remove redundant duplicate data from the wordlocation table

Solution:

Create a table wl_new with the same structure, and then execute the following statement

Insert into wl_new select * From wordlocationGroup by urlid, wordid

The last group by ensures the uniqueness of (urlid, wordid)

Delete all data in the wordlocation table, insert all data in wl_new to the wordlocation table, and delete wl_new.