Oracle Query Data deduplication method

SQL Duplicate Record Query method

August 14, 2008 Thursday 21:01

SQL Duplicate record query 1, look for redundant records in the table, duplicate records are based on a single field (Peopleid) to determine SELECT * from People where Peopleid in (select Peopleid from People GROUP by Peopleid have count (Peopleid) > 1) 2, delete redundant records in the table, duplicate records are based on a single field (Peopleid) to judge, leaving only the smallest ROWID records Delete from people where Peopleid in (select Peopleid from People GROUP by Peopleid have count (Peopleid) > 1) and rowID not in (select min (rowid) from people GROUP by Peopleid have count (Peopleid ) >1) 3. Find redundant duplicate records (multiple fields) in the table SELECT * FROM Vitae a where (A.PEOPLEID,A.SEQ) in (select Peopleid,seq from Vitae GROUP by PEOPLEID,SEQ have COUNT (*) > 1) 4. Delete extra duplicate records (multiple fields) in the table, leaving only the record with ROWID minimum Delete from Vitae a where (A.PEOPLEID,A.SEQ) in (select Peopleid,seq from Vitae GROUP by PEOPLEID,SEQ have COUNT (*) > 1) and rowID not in (select min (rowid) from Vitae GROUP by PEOPLEID,SEQ have Count (*) >1) 5. Find redundant duplicate records (multiple fields) in the table, not including the smallest ROWID records SELECT * FROM Vitae a where (A.PEOPLEID,A.SEQ) in (select Peopleid,seq from Vitae GROUP by PEOPLEID,SEQ have COUNT (*) > 1) and rowID not in (select min (rowid) from Vitae GROUP by PEOPLEID,SEQ have Count (*) >1) Two Say A field "name" exists in table A, and the "name" value may be the same between different records, Now is the need to query out the records in the table, "name" value has duplicate entries; Select Name,count (*) from A Group by Name have Count (*) > 1 If you also look at the same gender, the following is true: Select Name,sex,count (*) from A Group by Name,sex have Count (*) > 1 Three Method One Declare @max integer, @id integer Declare cur_rows cursor Local for select main field, COUNT (*) from table name Group by main Field having COUNT (*) >; 1 Open Cur_rows Fetch cur_rows into @id, @max While @ @fetch_status =0 Begin Select @max = @max-1 SET ROWCOUNT @max Delete from table name where main field = @id Fetch cur_rows into @id, @max End Close Cur_rows SET ROWCOUNT 0 Method Two There are two meanings of duplicate records, one is a completely duplicate record, that is, all fields are duplicated records, and second, some key fields Duplicate records, such as the Name field, and the other fields may not be repeated or repeated can be ignored. 1, for the first kind of repetition, easier to solve, using SELECT DISTINCT * from TableName You can get a result set with no duplicate records. If the table needs to delete duplicate records (duplicate records retain 1), you can delete them as follows SELECT DISTINCT * to #Tmp from TableName drop table TableName SELECT * Into TableName from #Tmp drop table #Tmp This duplication occurs because the table is poorly designed and the unique index columns are added to resolve. 2, this kind of repetition problem usually requires to keep the first record in the duplicate record, the operation method is as follows Suppose there is a duplicate field name,address, which requires the result set to be unique for both fields Select Identity (int,1,1) as Autoid, * into #Tmp from TableName Select min (autoid) as autoid into #Tmp2 from #Tmp Group by name,autoid SELECT * from #Tmp where autoid on (select Autoid from #tmp2) The last select is the result set that name,address not duplicate (but one more autoid field that can be written when actually writing Omit this column in the SELECT clause) (iv) Duplication of queries SELECT * FROM tablename where ID in ( Select ID from tablename GROUP BY ID Having count (ID) > 1 )

Oracle Query Data deduplication method