Handling Myloader Import data prompt primary key repeat issue

I. Description of the problem
Mydumper Backup of ABC and ABCB two libraries, using Myloader to import ABC database prompts primary key repeat error (many tables have this error):
* (myloader:15557): CRITICAL * *: Error restoring abcb.status_flow from File Abcb.status_flow.sql:Duplicate ' 10203 ' For key ' PRIMARY '
* * * (myloader:15557): CRITICAL * *: Error restoring Abcb.task_creation_master from File Abcb.task_creation_master.sql: Duplicate entry ' 280 ' for key ' PRIMARY '
The import script is as follows:
Myloader-h localhost-p 3306-u dba-p xxxx-s/tmp/mysql3306.sock-b abc-e-d/apps/mydumper_import/20171225010001-o -T 8
Version: Mariadb 10.1.18 + myloader 0.9.1

two. Problem Analysis
From the error point of view prompts the primary key to repeat, the first thought is the original library and table whether there is drop off, in fact, before the import has done a drop operation, there is no library and table did not drop off the problem, and check the import command is normal. Try to change to import ABCB Library, also prompts the primary key to repeat, guess the ABC library and ABCB library two libraries, the first three words of the same name, there will be conflict. or two libraries with the same table conflict. Then check the Mydumper backup file and sure enough to find two inventory in the same table name:
apps@hcp50019 20171225010001]$ Ls-al|grep Status_flow
-rw-r--r--. 1 Apps Apps 1546 Dec 17:37 Abcb.status_flow_detail-schema.sql
-rw-r--r--. 1 Apps Apps 162320 Dec 17:37 Abcb.status_flow_detail.sql
-rw-r--r--. 1 Apps Apps 1509 Dec 17:37 Abcb.status_flow-schema.sql
-rw-r--r--. 1 Apps Apps 19513 Dec 17:37 Abcb.status_flow.sql
-rw-r--r--. 1 Apps Apps 1546 Dec 17:37 Abc.status_flow_detail-schema.sql
-rw-r--r--. 1 Apps Apps 25078 Dec 17:37 Abc.status_flow_detail.sql
-rw-r--r--. 1 Apps Apps 1509 Dec 17:37 Abc.status_flow-schema.sql
-rw-r--r--. 1 Apps Apps 2825 Dec 17:37 Abc.status_flow.sql
View the Status_flow table definitions and structures under ABC and ABCB respectively:
Show CREATE TABLE Abcb.status_flow\g
1. Row ***************************
Table:status_flow
Create table:create Table ' Status_flow ' (
`........
PRIMARY KEY (' id '),
KEY ' idx1 ' (' Code ', ' Wcode ') USING btree
) Engine=innodb DEFAULT Charset=utf8
Show CREATE TABLE Abc.status_flow\g
1. Row ***************************
Table:status_flow
Create table:create Table ' Status_flow ' (
.......
PRIMARY KEY (' id '),
KEY ' idx1 ' (' Code ', ' Wcode ') USING btree
) Engine=innodb DEFAULT Charset=utf8
So guess for the Myloader import, the same table for two libraries was repeatedly imported, so the primary key was repeated.

Through the above comparative analysis, the reason for the error is roughly clear: when there are multiple libraries in an export backup, and when multiple inventory is in the same table name and table structure, the Myloader tool imports two libraries into one library, and the table structure in two libraries is the same, so this problem occurs.

Re-check the function of the-B parameter:

-B,--database an alternative database to restore into

The role here is to merge multiple libraries from the source into the libraries behind-B.

Three. Problem resolution view the parameters with the Myloader command:
Myloader--help
Application Options:
-D,--directory directory of the dump to import
-Q,--queries-per-transaction number of queries per transaction, default 1000
-O,--overwrite-tables Drop tables if they already exist
-B,--database an alternative database to restore into
-S,--source-db Database to restore
-E,--enable-binlog enable binary logging of the restore data
-H,--host the host to connect to
-U,--user Username with privileges to run the dump
-P,--password User password
-P,--port TCP/IP port to connect to
-S,--socket UNIX domain socket file to connection
-T,--threads number of threads to use, default 4
-C,--compress-protocol use compression on the MySQL connection
-V,--version show the program version and exit
-V,--verbose verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2

The-s parameter can specify the source database name, so after importing with the following command, the problem no longer occurs:

Myloader-h localhost-p 3306-u dba-p xxxx-s/tmp/mysql3306.sock-s db1-b db1-e-d/apps/mydumper_import/201712250100 01-o-T 8

Four. Summary

when importing a library from multiple repositories from a source to target MySQL, the-s and-B parameters are recommended in combination, and replication filtering is set from the library: set global REPLICATE_DO_DB=ABC.