"Go" backup Kylin metadata

http://blog.csdn.net/jiangshouzhuang/article/details/51290239

Kylin organizes all of its meta data (including cube descriptions and instances, projects, inverted index description and instances,jobs, tables and Dict ionaries) as a hierarchy of file systems.

However, Kylin uses HBase to store instead of the normal file system.

We can see from the Kylin configuration file kylin.properties:

# # The metadata store in HBase

[Email protected]

Metadata that represents Kylin is saved in the Kylin_metadata table of HBase.

Backing up the Kylin metadata

./bin/metastore.sh Backup

This will back up the metadata to the local directory KYLIN_HOME/METADATA_BACKPS below, the directory naming format is:

Kylin_home/meta_backups/meta_year_month_day_hour_minute_second

For example, my Kylin home directory is/var/lib/kylin/kylin, then the directory of backup data is:

/var/lib/kylin/kylin/meta_backups/meta_2016_05_01_11_50_32

Let's take a look at the catalogue:

[[Email protected]]$ cd /var/lib/kylin/kylin/meta_backups/meta_2016_05_01_11_50_32

[Email protected]_2016_05_01_11_50_32]$ ll

Total 44

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 Cube

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 cube_desc

Drwxrwxr-x 4 Kylin kylin 4096 May 1 11:50 cube_statistics

Drwxrwxr-x 6 Kylin kylin 4096 May 1 11:50 dict

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 Execute

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 execute_output

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 model_desc

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 project

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 table

Drwxrwxr-x 2 Kylin kylin 4096 May 1 11:50 table_exd

Drwxrwxr-x 5 Kylin kylin 4096 May 1 11:50 table_snapshot

Recovering metadata

If your Kylin metadata is dead, then we can use the previously backed up data for recovery:

1. First reset the current Kylin metadata store, which will clean up all kylin metadata stored in hbase, ensuring that backups have been made before

./bin/metastore.sh Reset

2. Next, upload the backup metadata to the Kylin metadata

./bin/metastore.sh restore$kylin_home/meta_backups/meta_xxxx_xx_xx_xx_xx_xx

Clean up useless resources from the Kylin meta-data

Over time, some resources, such as dictionaries, snapshots of tables, have become useless (cube's segment have been deleted or merged), but they still occupy space. You can perform the following command to find and clean up useless metadata:

1. First, perform a check, which is a safe operation and will not modify anything:

./bin/metastore.sh Clean

List the resources that need to be deleted

2. Next, in the above command, add the "--delete true" parameter, which will clean up any unused resources. Remember to back up the Kylin metadata before this command operation:

./bin/metastore.sh Clean--delete True

"Go" backup Kylin metadata