Mysql removes redundant duplicate records

Mysql deletes redundant duplicate records. Sometimes, multiple duplicate records exist in the database. In this case, deduplication is required, that is, only one record is retained... This is especially common in Data Warehouses because of human or even negligence. How to remove duplicates? The procedure is as follows: 1. query duplicate records? CREATETABLEpush_log_fu

Mysql deletes redundant duplicate records. Sometimes, multiple duplicate records exist in the database. In this case, deduplication is required, that is, only one record is retained... This is especially common in Data Warehouses because of human or even negligence. How to remove duplicates? The procedure is as follows: 1. query duplicate records? Create table push_log_fu

Mysql deletes redundant duplicate records

Sometimes, there are multiple duplicate records in the database. In this case, deduplication is required, that is, only one record is retained...

This is especially common in Data Warehouses because of human or even negligence.

How to remove duplicates?

The procedure is as follows:

1.

Query Repeated Records

?

CREATE TABLE push_log_full_2013_10_30_tmpSELECT * FROM `push_log_full`WHERE time BETWEEN FROM_DAYS(TO_DAYS(NOW()) - 1) AND FROM_DAYS(TO_DAYS(NOW()))AND (imsi, andriodid, time) IN (SELECT imsi, andriodid, time FROM `push_log_full`WHERE time BETWEEN FROM_DAYS(TO_DAYS(NOW()) - 1) AND FROM_DAYS(TO_DAYS(NOW()))GROUP BY imsi, andriodid, timeHAVING COUNT(*) > 1)

? Store duplicate records in a temporary table.

?

2

Delete the duplicate data in the push_log_full table.

3

Import the temporary table data to push_log_full.

?

?

The deduplication of Repeated Records is achieved through the above three steps.