Data migration practices

Winning record side

Data Volume: Tens

Implementation scenario: Database level migration, new data one by one match

Implementation scripts:

1, SQL sorting out the data needs to be exported:

Select A,b,c,d,e,f,g
 from Table_1 
                to outfile '/data/aaa/haha.csv ' fields
terminated by ', ' optionally Enclosed by ' \ '
lines terminated by ' \ r \ n ';
 

2, the use of shell segmentation record results, batch Import

Split-a 3-l 20000 haha.csv target/table2_ for
  
i to ' ls-1 target '
do
    mysql-s-n-p ' xxxxxxx ' Dpemidas-e ' L Oad data infile '/data/aaa/target/$i '
IGNORE into table table2 fields terminated
by ', '  optionally enclosed By ' \ ' '
lines terminated by ' \ r \ n '
    mv/data/aaa/target/$i/data/aaa/complete/sleep
    1
done