One step to complete MySQL migration to Redis

Moving a big table from MySQL to Redis, you'll find that the slow speed you get when extracting, converting, or loading a row of data is unbearable. I'm going to tell you a little trick to get you out of here. Use the "Pipeline output" method to pass the contents of the MySQL command line directly to the REDIS-CLI, in order to bypass the "middleware" way to make the two in the data operation to achieve the best speed. A MySQL table of about 8 million rows of data, originally imported into Redis takes 90 minutes, and it takes only two minutes to use this method. Whether you believe it or not, I believe it anyway. Zzxworldzzxworld translated 3 years ago 14 people top top translation Good Oh! Other translations (1) MySQL to redis Data protocol REDIS-CLI command-line tool has a bulk insert mode that is specifically designed for batch execution commands. The first step is to format the contents of the MySQL query into REDIS-CLI available data formats. Here we go!zzxworldzzxworld translated 3 years ago 5 people top top translation Good Oh! My statistics:? 1234567CREATE TABLE events_all_time (id int (one) unsigned not NULL Auto_increment, Action varchar (255) NOT NULL, Count Int (one) not null DEFAULT 0, PRIMARY key (ID), UNIQUE key Uniq_acti On (action); The Redis command that prepares to execute in each row of data is as follows:? 1HSET Events_all_time [Action] [count] Follow the Redis command rules above to create a events_to_ Redis.sql file that is used to generate the Redis data protocol format sql:?123456789101112131415161718192021--Events_to_redis.sql select CONCAT ("*4\r\n ", ' $ ', Length (redis_cmd), ' \ r \ n ', Redis_cmd, ' \ r \ n ', ' $ ', Length (redis_key), ' \ r \ n ', Redis_key, ' \ r \ n ', ' $ ', LENGTH ( hkey), ' \ r \ n ', hkey, ' \ r \ n ', ' $ ', LENGTH (hval), ' \ r \ n ', Hval, ' \ R ') from (SElect ' Hset ' as Redis_cmd, ' events_all_time ' as Redis_key, action as HKEY, count as Hval from Events_all_time) as Tok , execute with the following command:? 1mysql stats_db--skip-column-names--raw < Events_to_redis.sql | REDIS-CLI--pipe Important MySQL parameter description:--raw: Make MySQL do not convert line breaks in field values. --skip-column-names: Does not include the column name in each line of MySQL output. Zzxworldzzxworld translated 3 years ago 6 people top top translation Good Oh! All translations in this article are for learning and communication purposes only, please note that the translator, source, and this article link our translation work in accordance with the CC agreement, if our work has violated your rights and interests, please contact us promptly

　　

One step to complete MySQL migration to Redis