MySQL查詢、刪除重複記錄

表結構，數據以下：

CREATE TABLE `redis_num_mem` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
  `cluster_id` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'Redis集羣id',
  `type` varchar(10) NOT NULL DEFAULT '' COMMENT 'key類型',
  `number` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'key的個數',
  `memory` bigint(20) unsigned NOT NULL DEFAULT 0 COMMENT 'key的存內，單位bytes',
  `create_time` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT '記錄生成時間',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=103 DEFAULT CHARSET=utf8 COMMENT='redis類型表'


MariaDB [test]> select * from redis_num_mem;
+-----+------------+--------+--------+--------+---------------------+
| id  | cluster_id | type   | number | memory | create_time         |
+-----+------------+--------+--------+--------+---------------------+
|   1 |          1 | hash   |  47079 |    100 | 2018-10-22 23:48:12 |
|   2 |          1 | set    |  20608 |    100 | 2018-10-22 23:48:12 |
|   3 |          1 | list   |   3938 |    100 | 2018-10-22 23:48:12 |
|   4 |          1 | string |  17662 |    100 | 2018-10-22 23:48:12 |
|   5 |          1 | hash   |  46808 |    100 | 2018-10-22 23:48:12 |
|   6 |          1 | set    |  20854 |    100 | 2018-10-22 23:48:12 |
|   7 |          1 | string |  17710 |    100 | 2018-10-22 23:48:12 |
|   8 |          1 | list   |   3917 |    100 | 2018-10-22 23:48:12 |
|   9 |          1 | list   |   3854 |    100 | 2018-10-22 23:48:12 |
|  10 |          1 | string |  17651 |    100 | 2018-10-22 23:48:12 |
|  11 |          1 | hash   |  46960 |    100 | 2018-10-22 23:48:12 |
|  12 |          1 | set    |  20654 |    100 | 2018-10-22 23:48:12 |
|  14 |          2 |        |      0 |      0 | 2018-10-23 10:34:54 |
|  15 |          2 |        |      0 |      0 | 2018-10-23 10:35:42 |
|  16 |          0 |        |      0 |      0 | 2018-10-23 10:50:39 |
|  20 |          0 |        |      0 |      0 | 2018-10-23 11:05:52 |
| 100 |          0 |        |      0 |      0 | 2018-10-23 11:05:38 |
| 101 |          0 |        |      0 |      0 | 2018-10-23 11:06:09 |
+-----+------------+--------+--------+--------+---------------------+
18 rows in set (0.00 sec)

查找全部重複類型的記錄：

SELECT * FROM redis_num_mem a WHERE ((SELECT COUNT(*) FROM redis_num_mem WHERE type = a.type) > 1) ORDER BY type DESC;

1、查找重複記錄

1.1  查找所有重複記錄

Select * From redis_num_mem Where type In (Select type From redis_num_mem Group By type Having Count(*)>1) ;

1.2  過濾重複記錄(只顯示一條)

select * From redis_num_mem where id in (Select Max(id) From  redis_num_mem Group By type);

注：此處顯示ID最大一條記錄

2、刪除重複記錄

2.1  刪除所有重複記錄（慎用）

delete from redis_num_mem where type in (select type from (select type from redis_num_mem group by type having count(*)>1)  as t1);

2.2  保留一條（這個應該是大多數人所須要的 ^_^）

delete from redis_num_mem where id not in (select * from (select max(id) from redis_num_mem group by type) as t1);

3、補充

一、查找表中多餘的重複記錄，重複記錄是根據單個字段（peopleId）來判斷

select * from people where peopleId in (select peopleId from people group by peopleId having count(peopleId) > 1)

 

二、刪除表中多餘的重複記錄，重複記錄是根據單個字段（peopleId）來判斷，只留有rowid最小的記錄

delete from people where peopleId in (select peopleId from people group by peopleId having count(peopleId) > 1) and rowid not in (select min(rowid) from people group by peopleId having count(peopleId )>1)

 

三、查找表中多餘的重複記錄（多個字段）

select * from vitae a where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)

 

四、刪除表中多餘的重複記錄（多個字段），只留有rowid最小的記錄

delete from vitae a where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1) and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)

 

五、查找表中多餘的重複記錄（多個字段），不包含rowid最小的記錄

select * from vitae a where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1) and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)