Pymysql.err.IntegrityError: (1062, "Duplicate entry ' roxml-roxml ' for key ' PRIMARY '")

In the 3rd entity match in Python data mining-concepts, methodologies and practices, the error shown in the topic appears if you run the configuration in accordance with the author's code and SQL statements. According to the execution error prompt of the Python script, the error occurred in the first statement in the scriptINSERT INTO.

In fact, not to insertroxmltime will be error, but in frontoptionparserof the insertion time has been an error. This error means that the code tries to insert two duplicate values in the primary key column of the table. The problem took me three hours to finally solve it.

The first statement in the book is to insert the project name of theINSERTproject with the same URL into thebook_entity_matchestable, in the project of two websites from RF and RG, there are some items whose names are only case-sensitive, as follows:

In the two figure, which is the result of the query from the RF and RG tables, it is true that there are items with the project name that differ only in case. But because MySQL defaults to its varchar field is case-insensitive,INSERT INTOAn error occurs when executing a statement when trying to insert an item name that is the same as shown in the two identical (case-only difference)!

One of the workarounds is to change the two field types to binary:

ALTER TABLE book_entity_matches MODIFY COLUMN rg_project_name VARCHAR(100) BINARY NOT NULL;
ALTER TABLE book_entity_matches MODIFY COLUMN rf_project_name VARCHAR(100) BINARY NOT NULL;

Other methods include specifying that the field property is binary when the table is created, or specifying the encoding type when the table is created, and that the type suffix cs (such as Utf8_general_cs) is case-sensitive and is not differentiated for CI (such as utf8_general_ci).

Reference:

1. MYSQL setting field values are case sensitive
2. MySQL Case sensitive description

Pymysql.err.IntegrityError: (1062, "Duplicate entry ' roxml-roxml ' for key ' PRIMARY '")