Omnipitr of PostgreSQL Backup

Source: Internet
Author: User
Tags postgresql postgresql backup git clone

First, prepare1. Installation

Omnipitr is written in Perl and can be downloaded directly.

# git clone https://github.com/omniti-labs/omnipitr.git/opt/omnipitr/Initialized empty git repository in/opt/omnipitr /.git/remote:counting objects:2627, done.remote:Total 2627 (Delta 0), reused 0 (Delta 0), pack-reused 2627Receiving obj ects:100% (2627/2627), 742.71 KiB | KIB/S, done. Resolving deltas:100% (1287/1287), done.
2. Check:
#/opt/omnipitr/bin/sanity-check.shchecking:-/opt/omnipitr/bin-/opt/omnipitr/lib9 programs, libraries. Tar Versionall checked, and looks OK.
3. Create a directory:
# mkdir-p/data/omnipitr/test_94/{log,state,tmp}# chown-r Postgres:postgres/data/omnipitr
Second, backup1. Backing up from the master node

1) Modify Archive_command

Test! -F/DATA/OMNIPITR/TEST_94/TMP/BACKUP/%F && CP%p/data/omnipitr/test_94/tmp/backup/%f

If you have archived the Wal elsewhere, you do not need to back up the Wal file, you do not need to configure this

2) back up to local

$/opt/omnipitr/bin/omnipitr-backup-master--host=localhost \ #master节点ip--username=postgres \ #连接master节点的用户--port= 5432 \ #master节点的端口--data-dir=/data/postgres/data/test_94/\ #master节点的数据目录--dst-local gzip=/data/postgres/backup/t est_94/\ #本地备份目录--xlogs=/data/omnipitr/test_94/tmp/backup \ #用于归档xlog的目录, cannot exist before executing the command, if backup xlog is not required, Replace with--skip-xlogs--temp-dir=/data/omnipitr/test_94/tmp/\ #创建临时文件的目录, can not be specified, default is/tmp--pid-file=/data/omnipitr/test_    94/STATE/BASE_BACKUP.PID \ #进程文件 To ensure that only one backup is performed--log=/data/omnipitr/test_94/log/base_backup.log \ #日志输出文件--verbose #打印详细信息

3) verbose log information for backup

2015-09-09 04:50:54.185227 -0400 : 9766 : omnipitr-backup-master : log  : called with parameters: --host=localhost --username=postgres --port=5432  --data-dir=/data/postgres/data/test_94/ --dst-local gzip=/data/postgres/backup/test_94/ - -xlogs=/data/omnipitr/test_94/tmp/backup --temp-dir=/data/omnipitr/test_94/tmp/ --pid-file=/data/ omnipitr/test_94/state/base_backup.pid --log=/data/omnipitr/test_94/log/base_backup.log -- verbose2015-09-09 04:50:54.363777 -0400 : 9766 : omnipitr-backup-master :  log : timer [select w, pg_xlogfile_name (W)  from  (Select pg_start _backup (' Omnipitr ')  as w )  as x] took: 0.175s2015-09-09  04:50:54.366482 -0400 : 9766 : omnipitr-backup-master : log : pg_ Start_backup (' Omnipitr ')  returned 1/3c000028|00000001000000010000003c.2015-09-09 04:50:54.368554 -0400 : 9766 :  omnipitr-backup-master : log : script to make tarballs:2015-09-09  04:50:54.368554 -0400 : 9766 : omnipitr-backup-master : log :  mkfifo \/tmp\/commandpiper\-9766\-ldy5al\/fifo\-02015-09-09 04:50:54.368554 -0400 :  9766 : omnipitr-backup-master : LOG : nice gzip \-\-stdout  \- < \/tmp\/commandpiper\-9766\-ldy5al\/fifo\-0 > \/data\/postgres\/backup\/test_94\ /qa\-5\-45\-data\-2015\-09\-09\.tar\.gz &2015-09-09 04:50:54.368554 -0400 : 9766  : omnipitr-backup-master : log : nice tar cf \- \-\-exclude \=pg_log\/\* \-\-exclude\=pg_xlog\/0\* \-\-exclude\=pg_xlog\/archive_status\/\* \-\-exclude\= Postmaster\.pid tesT_94 2> \/data\/omnipitr\/test_94\/tmp\/omnipitr\-backup\-master\/9766\/tar\.stderr >  \/tmp\/commandpiper\-9766\-ldy5al\/fifo\-02015-09-09 04:50:54.368554 -0400 : 9766  : omnipitr-backup-master : log : wait2015-09-09 04:50:54.368554 -0400  : 9766 : omnipitr-backup-master : log : rm \/tmp\/commandpiper\- 9766\-ldy5al\/fifo\-02015-09-09 04:51:14.415299 -0400 : 9766 :  omnipitr-backup-master : log : timer [compressing  $PGDATA] took:  20.048s2015-09-09 04:51:15.483531 -0400 : 9766 : omnipitr-backup-master :  log : timer [select pg_stop_backup ()] took: 1.066s2015-09-09  04:51:15.486756 -0400 : 9766 : omnipitr-backup-master : log : pg_ Stop_backup (' Omnipitr ')  returned 1/3c000128.2015-09-09 04:51:15.487734 -0400 : 9766 : omnipitr-backup-master :  LOG : Timer [Making data archive] took: 21.300s2015-09-09  04:51:15.489015 -0400 : 9766 : omnipitr-backup-master : log :  file 00000001000000010000003c.00000028.backup arrived after 0 seconds.2015-09-09  04:51:15.490275 -0400 : 9766 : omnipitr-backup-master : log :  File 00000001000000010000003C arrived after 0 seconds.2015-09-09  04:51:15.491816 -0400 : 9766 : omnipitr-backup-master : log :  script to make tarballs:2015-09-09 04:51:15.491816 -0400 : 9766 :  omnipitr-backup-master : log : mkfifo \/tmp\/commandpiper\-9766\-ldy5al\/fifo\- 02015-09-09 04:51:15.491816 -0400 :  9766 : omnipitr-backup-master : log : nice gzip \-\-stdout  \- < \/tmp\/commandpiper\-9766\-ldy5al\/fifo\-0 > \/data\/postgres\/backup\/ test_94\/qa\-5\-45\-xlog\-2015\-09\-09\.tar\.gz &2015-09-09 04:51:15.491816 -0400 :  9766 : omnipitr-backup-master : LOG : nice tar cf \-  Test_94 2> \/data\/omnipitr\/test_94\/tmp\/omnipitr\-backup\-master\/9766\/tar\.stderr >  \/tmp\/commandpiper\-9766\-ldy5al\/fifo\-02015-09-09 04:51:15.491816 -0400 : 9766  : omnipitr-backup-master : log : wait2015-09-09 04:51:15.491816 -0400  : 9766 : omnipitr-backup-master : log : rm \/tmp\/commandpiper\- 9766\-ldy5al\/fifo\-02015-09-09 04:51:15.936762 -0400 : 9766 :  Omnipitr-backup-master : log : timer [compressing xlogs] took: 0.445s2015-09-09 04:51:15.947223 -0400  : 9766 : omnipitr-backup-master : log : timer [making xlog  archive] took: 0.458s2015-09-09 04:51:15.948752 -0400 : 9766 :  omnipitr-backup-master : log : timer [delivering to all remote  destinations] took: 0.000s2015-09-09 04:51:15.950304 -0400 : 9766 :  Omnipitr-backup-master : log : timer [delivering meta files] took:  0.001s2015-09-09 04:51:15.951272 -0400 : 9766 : omnipitr-backup-master  : log : timer [whole backup procedure] took: 21.764s2015-09-09  04:51:15.952141 -0400 : 9766 : omnipitr-backup-master : log :  all done.

4) Backing up files

$ ll/data/postgres/backup/test_94/total 95620-rw-rw-r--. 1 postgres postgres 97851471 Sep 9 04:56 qa-5-45-data-2015-09-09.tar.gz-rw-rw-r--. 1 Postgres postgres, Sep 9 04:56 qa-5-45-meta-2015-09-09.tar.gz-rw-rw-r--. 1 postgres postgres 54742 Sep 9 04:56 qa-5-45-xlog-2015-09-09.tar.gz

5) Backup to remote

$/opt/omnipitr/bin/omnipitr-backup-master--host=localhost--username=postgres--port=5432--data-dir=/data/ postgres/data/test_94/--dst-direct [email protected]:/data/postgres/backup/test_94/\ #使用ssh方式传送备份--skip-xlogs \ #不 Backing up a Wal file--temp-dir=/data/omnipitr/test_94/tmp/--pid-file=/data/omnipitr/test_94/state/base_backup.pid--log=/ Data/omnipitr/test_94/log/base_backup.log--verbose [email protected] ' s password:
2. Backup from slave node

1) Backup

$/opt/omnipitr/bin/omnipitr-backup-slave--host=172.17.5.45 \ #master节点ip--username=postgres--port=5432-- Call-master \ #保证在master上执行SELECT pg_start_backup (' ... '); and select Pg_stop_backup (), otherwise the backup made out may be invalid--data-dir=/data/postgres/data/test_94/--dst-local gzip=/data/postgres/ backup/test_94/--skip-xlogs--temp-dir=/data/omnipitr/test_94/tmp/--pid-file=/data/omnipitr/test_94/state/base_ Backup.pid--log=/data/omnipitr/test_94/log/base_backup.log--verbosepassword for the user Postgres:password for user Postgres:password for user postgres:

2) BACKUP Log

2015-09-09 06:16:55.621897 -0400 : 23447 : omnipitr-backup-slave : log  : called with parameters: --host=172.17.5.45 --username=postgres --port= 5432 --call-master --data-dir=/data/postgres/data/test_94/ --dst-local gzip=/data/postgres /backup/test_94/ --skip-xlogs --temp-dir=/data/omnipitr/test_94/tmp/ --pid-file=/data/omnipitr/ test_94/state/base_backup.pid --log=/data/omnipitr/test_94/log/base_backup.log --verbose2015-09-09  06:16:58.856673 -0400 : 23447 : omnipitr-backup-slave : log :  timer [select w, pg_xlogfile_name (W)  from  ( as w )  as  x] took: 3.220s2015-09-09 06:16:58.864733 -0400 : 23447 :  Omnipitr-backup-slave : log : pg_start_backup (' Omnipitr ',  true)  returned 1/ 62000028|000000010000000100000062.2015-09-09 06:17:01.862033 -0400 : 23447 : omnipitr-backup-slave :  log : timer [select pg_read_file (  ' Backup_label ', 0,  ( pg_stat_ File (  ' Backup_label '  )  ) .size )] took: 2.996s2015-09-09 06:17:01.863971  -0400 : 23447 : omnipitr-backup-slave : log : waiting for  checkpoint  (Based on backup_label from master)  - CHECKPOINT  location: 1/620000602015-09-09 06:18:37.008788 -0400 : 23447 :  omnipitr-backup-slave : log : checkpoint .2015-09-09 06:18:37.013235 -0400  : 23447 : omnipitr-backup-slave : log : script to make  tarballs:2015-09-09 06:18:37.013235 -0400 : 23447 : omnipitr-backup-slave  : log : mkfifo \/tmp\/commandpiper\-23447\-bup6d5\/fifo\-02015-09-09 06:18:37.013235 -0400 : 23447 :  omnipitr-backup-slave : LOG : nice gzip \-\-stdout \- <  \/tmp\/commandpiper\-23447\-bup6d5\/fifo\-0 > \/data\/postgres\/backup\/test_94\/node2\-data\- 2015\-09\-09\.tar\.gz &2015-09-09 06:18:37.013235 -0400 : 23447 :  Omnipitr-backup-slave : log : nice tar cf \- \-\-exclude\=test_94\/pg_ log\/\* \-\-exclude\=test_94\/pg_xlog\/0\* \-\-exclude\=test_94\/pg_xlog\/archive_status\/\* \- \-exclude\=test_94\/recovery\.conf \-\-exclude\=test_94\/postmaster\.pid \-\-transform\=s\#\^data\/ Omnipitr\/test_94\/tmp\/omnipitr\-backup\-slave\/23447\/\ #test_94 \/\# test_94 \/data\/omnipitr\/test _94\/tmp\/omnipitr\-backup\-slave\/23447\/backup_label 2> \/data\/omnipitr\/test_94\/tmp\/ omnipitr\-backup\-slave\/23447\/tar\.stderr > \/tmp\/commandpiper\-23447\-bup6d5\/fifo\-02015-09-09 06:18:37.013235  -0400 : 23447 : omnipitr-backup-slave : log : wait2015-09-09  06:18:37.013235 -0400 : 23447 : omnipitr-backup-slave : log : rm  \/tmp\/commandpiper\-23447\-bup6d5\/fifo\-02015-09-09 06:18:57.288130 -0400 : 23447  : omnipitr-backup-slave : log : tar stderr:2015-09-09 06:18:57.290137  -0400 : 23447 : omnipitr-backup-slave : log : ================== ============================2015-09-09 06:18:57.290849 -0400 : 23447 :  omnipitr-backup-slave : log : tar: removing leading  '/'  from  member names2015-09-09 06:18:57.291659 -0400 : 23447 :  omnipitr-backup-slave : log :  ==============================================2015-09-09 06:18:57.292502 -0400 : 23447  : omnipitr-backup-slave : LOG : Timer [Compressing  $PGDATA] took :  20.280s2015-09-09 06:19:34.578726 -0400 : 23447 : omnipitr-backup-slave  : log : timer [select pg_stop_backup ()] took: 37.285s2015-09-09  06:19:34.580554 -0400 : 23447 : omnipitr-backup-slave : log : pg_ Stop_backup ()  returned 1/62000128.2015-09-09 06:19:34.581542 -0400 : 23447  :  omnipitr-backup-slave : log : timer [making data archive] took:  158.950s2015-09-09 06:19:34.582615 -0400 : 23447 : omnipitr-backup-slave  : LOG : Timer [Making xlog archive] took: 0.000s2015-09-09  06:19:34.583894 -0400 : 23447 : omnipitr-backup-slave : log : timer [ delivering to all remote destinations] took: 0.000s2015-09-09  06:19:34.585621 -0400 : 23447 : omnipitr-backup-slave : log :  timer [delivering meta files] took: 0.000s2015-09-09 06:19:34.586895 -0400  : 23447 : omnipitr-backup-slave : log : timer [whole backup  procedure] took: 158.955s2015-09-09 06:19:34.587940 -0400 : 23447 :  omnipitr-backup-slave : log : all done.

3) Backing up files

# ll/data/postgres/backup/test_94/total 95548-rw-r--r--. 1 postgres postgres 97836713 Sep 9 06:18 node2-data-2015-09-09.tar.gz-rw-r--r--. 1 postgres postgres Sep 9 06:19 node2-meta-2015-09-09.tar.gz

4) Remark

The default execution Pg_start_backup () is not true, and the red part of it is that I modified the code so that the execution of checkpoint would be much faster


Omnipitr of PostgreSQL Backup

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.