Troubleshoot Maxwell can not find database and use MySQL Binlog to solve related problems

We are currently using Maxwell to synchronize our offline database with the Binlog of the machine on the read line.

In this error location, the first line is to determine whether the problem occurs in the producer or the queue or the consumer. After looking at the running logs of the tasks on each machine, the problem is identified on the producer of the task.

So I tried to look at the Maxwell logs on the producers. It's been a good run before, Maxwell. The error log is a piece of content:

17:53:24,420 INFO  OpenReplicator - starting replication at mysql-bin.000055:17870889
17:53:24,944 INFO  TaskManager - stopping 3 tasks 17:53:24,944 ERROR TaskManager - cause:
com.zendesk.maxwell.schema.ddl.InvalidSchemaError: Couldn‘t find database ‘remain_db‘  at com.zendesk.maxwell.schema.Schema.findDatabaseOrThrow(Schema.java:52) ~[maxwell-1.10.3.jar:1.10.3]
    at com.zendesk.maxwell.schema.ddl.TableCreate.resolve(TableCreate.java:34) ~[maxwell-1.10.3.jar:1.10.3]
    at com.zendesk.maxwell.schema.ddl.TableCreate.resolve(TableCreate.java:13) ~[maxwell-1.10.3.jar:1.10.3]

Can see Maxwell try to read MySQL binlog.00055:17870889 This position content is need to find a thing called remain_db, because there is no remain_db so this behavior failed, resulting in the entire synchronous production The person is stuck.

To troubleshoot this problem, one thing we need to know is that Maxwell will proactively build a database of its own on a synchronized database called Maxwell like this:

The Maxwell database will record all kinds of information that will be needed when synchronizing, and when he synchronizes to the main database, he will use the relevant statements to build these things in his own database. Here the hint can not find remain_db is because Maxwell own database maintenance databases table inside no remain_db This database record, so the error.

Why are there missing DDL when normal maintenance? Think about it. This should not happen if the Binlog has not had a problem. So if you think it should be easy to locate the problem in the Binlog log above. Later I passed the MySQL configuration file view configuration when the problem is found, it is our database specifies that only a certain database synchronization, but before someone in the database use a library when the other library created operations, resulting in this creation behavior of the DDL is incorrectly synchronized to the user The Binlog of the table also led to this error.

It is relatively easy to find the error to solve, because the error message and the Maxwell data table clearly records the binlog of the interrupt location, I just need to go to the interrupt position to skip the relevant statement.

Specific Binlog View Operation command can refer to the contents of reference here only a few of the most practical, such as I will be practical

mysqlbinlog --start-position = 205 --stop-position = 314 mysql-bin.000001
To specify the start and end positions to get the intermediate binlog information

I can also use mysqlbinlog mysql-bin.0000001 | grep ‘xxxx’ to get the information I follow 

You can also use similar statements inside the MySQL shell:

 from 4 limit 2;

To view the relevant Binlog logs by looking at the specified position location and specifying how many counts.

Reference:

Http://soft.dog/2016/06/13/dig-mysql-binlog/#%E5%91%BD%E4%BB%A4%E6%B1%87%E6%80%BB MySQL Binlog view method

Troubleshoot Maxwell can not find database and use MySQL Binlog to solve related problems