Python -- boto3: basic interaction with dynamoDB, table backup and recovery, and pythondynamodb

Python -- boto3: basic interaction with dynamoDB, table backup and recovery, and pythondynamodb

References: http://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html

Recently, I have studied dynamoDB in boto3 due to work needs. I would like to summarize my experiences here.

First, install boto3 and run it on a machine with python and pip installed.

sudo pip install boto3  

 

Boto3 provides the following interfaces for interacting with dynamoDB:

batch_get_item()batch_write_item()can_paginate()create_table()delete_item()delete_table()describe_limits()describe_table()describe_time_to_live()generate_presigned_url()get_item()get_paginator()get_waiter()list_tables()list_tags_of_resource()put_item()query()scan()tag_resource()untag_resource()update_item()update_table()update_time_to_live()

To put it bluntly, we add, delete, query, and modify tables and records. This article mainly describes the interfaces I recently used.

 

To use boto3 in python, you must first import boto3. Of course, this is nonsense. For ease of use, I first wrote a configuration file in json format, as shown below:

{    "region_name":"xxx",    "aws_access_key_id":"xxx",    "aws_secret_access_key":"xxx"}

 

Then, it encapsulates a class specifically used to operate dynamoDB. Currently, nothing exists.

class dynamodb_operation():

It requires a method to read json files:

    def load_json(self,path):        try:            with open(path) as json_file:                data = json.load(json_file)        except Exception as e:            print 'ERROR: no such file like ' + path            exit(-1)        else:            return data

Because the read file may not be in json format, I want him to report an error and exit. If you don't want to let it exit, you can change it in commit T.

Then, I want this class to have a private member client that establishes a connection when I instantiate the object. Therefore, the following initialization method is available:

    def __init__(self,path):        conf = self.load_json(path)        self.client = boto3.client('dynamodb',region_name=conf['region_name'],aws_access_key_id=conf['aws_access_key_id'], aws_secret_access_key=conf['aws_secret_access_key'])

It corresponds to the previous configuration file.

With this foundation, you can encapsulate the methods you want to use. The instructions on the official website of each method will not be copied.

1. List all tables in dynamoDB

    def list_all_table(self):        page=1        LastEvaluationTableName = ""        while True:            if page == 1:                response = self.client.list_tables()            else:                response = self.client.list_tables(                        ExclusiveStartTableName=LastEvaluationTableName                        )            TableNames = response['TableNames']            for table in TableNames:                print table            if response.has_key('LastEvaluatedTableName'):                LastEvaluationTableName = response["LastEvaluatedTableName"]            else:                break            page += 1

The list_table () method can obtain a maximum of 100 table names at a time, and the value of key "LastEvaluatedTableName" for each return is the name of the last table, it can be used as a parameter for the next request. In this way, you can obtain all the table names. If no table is available, there will be no LastEvaluatedTableName in response. Here, I just want to print the table name to the terminal. If I want to save it, I can.

2. Get the information about a table: describe_table ()

    def get_table_desc_only(self,table):        try:            response = self.client.describe_table(TableName=table)        except Exception as e:            print 'ERROR: no such table like ' + table            exit(-1)        else:            return response["Table"]

In this example, only response ["Table"] is returned in original order without any additional processing.

If you want to know the size of a table, you can:

    def get_table_size(self,table):        response = self.get_table_desc_only(table)        stastic = {}        stastic['TableSizeBytes'] = response['TableSizeBytes']        stastic['ItemCount'] = response['ItemCount']        return stastic

If you want to know other information and only want to know that information, you can also write the corresponding method.

3. Create a table

    def create_table(self,tablename,keySchema,attributeDefinitions,provisionedThroughput):        table = self.client.create_table(                TableName=tablename,                KeySchema=keySchema,                AttributeDefinitions=attributeDefinitions,                ProvisionedThroughput=provisionedThroughput                )        # Wait until the table exists.        self.client.get_waiter('table_exists').wait(TableName=tablename)        response = self.client.describe_table(TableName=tablename)        print response

This is creating a table without indexes. It takes time to create a table, so the get_waiter () method is used.

4. insert data

    def put_item(self,tableName,item):        try:            self.client.put_item(                    TableName=tableName,                    Item=item                    )        except Exception as e:            print 'ERROR: put item fail. msg: ' + str(e)            exit(-1)        else:            return

Encapsulation of this method requires the input of a json in the correct format, and the key must correspond to the table. For example:

{'uid':{'N':'999'},'aid':{'N':'999'},'sid':{'N':'999'},'ksid':{'N':'999'}}

5. delete a table

    def delete_table(self,table):        try:            self.client.delete_table(                    TableName=table                    )           except Exception as e:            print 'ERROR: delete table ' + table + ' fail. msg: ' + str(e)        else:            print 'delete table ' + table + ' succ'

 

To be continued .....