Development and use of NOVA database modules __ Database

In nova.db this piece of e\f\g\h these versions are not very different, but starting from the G version to join the conductor, does not allow compute direct access to the database, so in the compute code to call the database through conductor.
If you want to add a new feature, moreover this function needs to operate the database, in the operation database this aspect generally divides into two steps: First, the DB module content compilation, mainly includes the data table creation, the function and the API compilation; two, in the Compute module, the DB provides the API invocation method to write.

Recommended Priority Learning SQLAlchemy module

OpenStack Environment Version: H version (G version similar, f\e version of the difference will be explained)

Content writing in DB module

1. Description data table
Nova Database creation:
When the Nova database is synchronized for the first time:

Nova-manage db Sync

This specified code is in nova.db.sqlalchemy.migration.py:

def db_sync (version=none):
    If version is not None:
        try:
            Version = Int (version)
        except ValueError:
            Raise exception. Novaexception (_ ("version should is an integer")

    current_version = db_version ()
    repository = _find_migrate_ Repo ()
    if version are None or version > current_version: Return
        Versioning_api.upgrade (Get_engine (), Repository, Version)
    else: return
        Versioning_api.downgrade (Get_engine (), Repository,
                                        version)

Because Version=none, so go to upgrade, it will eventually find nova.db.sqlalchemy.migrate_repo.versions.133_ Folsom.py The upgrade method in this file, the code is very long, here only the part that creates the data table is intercepted:
First, write the structure of a data table (the following is the Nova.instances table structure):

    instances = Table (' instances ', Meta, column (' Created_at ', datetime), column (' Updated_at ', datetime),  Column (' Deleted_at ', DateTime), column (' deleted ', Boolean), column (' ID ', Integer, Primary_key=true, NULLABLE=FALSE), column (' internal_id ', Integer), column (' user_id ', String (length=255)), column (' P Roject_id ', String (length=255)), column (' Image_ref ', String (length=255)), column (' kernel_id ', String (lengt h=255)), column (' ramdisk_id ', String (length=255)), column (' server_name ', String (length=255)), Colu
        Mn (' Launch_index ', Integer), column (' Key_name ', String (length=255)), column (' Key_data ', Mediumtext ()), 
        Column (' Power_state ', Integer), column (' Vm_state ', String (length=255)), column (' Memory_mb ', Integer), Column (' Vcpus ', Integer), column (' hostname ', string (length=255)), column (' Host ', String (length=25 5)), Column (' useR_data ', Mediumtext ()), column (' reservation_id ', String (length=255)), column (' Scheduled_at ', DateTime), Column (' Launched_at ', datetime), column (' Terminated_at ', datetime), column (' Display_name ', String (le ngth=255)), column (' Display_description ', String (length=255)), column (' Availability_zone ', String (length=2 ), column (' Locked ', Boolean), column (' Os_type ', String (length=255)), column (' Launched_on ', Medi Umtext ()), column (' instance_type_id ', Integer), column (' Vm_mode ', String (length=255)), column (' Uui d ', String (length=36)), column (' Architecture ', String (length=255)), column (' Root_device_name ', String (Leng
        th=255)), column (' Access_ip_v4 ', String (length=255)), column (' Access_ip_v6 ', String (length=255)), Column (' Config_drive ', String (length=255)), column (' Task_state ', String (length=255)), column (' Default_ephe Meral_device ', String (length=255)), column (' Default_swap_device ', String (length=255)), column (' Progress ', Integer), column (' Auto_disk_config ', Boolean), column (' Shutdown_terminate ', Boolean), column (' Disable_terminate ', Boolean) , column (' Root_gb ', Integer), column (' Ephemeral_gb ', Integer), column (' Cell_name ', String (length=2
 )), mysql_engine= ' InnoDB ', mysql_charset= ' UTF8 ')

To create a table:

    tables = [... ..., instances, ....]
    
    For table in tables:
        try:
            table.create ()
        except Exception:
            log.info (repr (table))
            log.exception (_ (' Exception while creating table. '
            ) Raise

In other words, if you want to create a new table in Nova-manage db Sync, you need to add a table structure description to the upgrade method, and then add the table name to the list element.

The establishment of 2.model
Assuming you have created this table, it is now necessary to write an API for the table.
Open nova.db.sqlalchemy.models.py, which is used to define the data table model, and to convert a data table into a class.
Take the model of snapshot as an example:

Class Snapshot (BASE, Novabase): "" "" "
    represents a block storage device the can is attached to a VM.
    " " __tablename__ = ' snapshots '
    __table_args__ = ()
    id = Column (String), Primary_key=true, Nullable=false)
    deleted = Column (String, default= "")

    @property
    def name (self): return
        conf.snapshot_name_template% Self.id

    @property
    def volume_name (self): return
        conf.volume_name_template% self.volume_id

    user_ id = column (string (255))
    project_id = Column (string (255))

    volume_id = Column (string), Nullable=false)
    status = Column (string (255))
    progress = column (string (255))
    volume_size = column (Integer)
    scheduled_at = Column (DateTime)

    display_name = Column (string (255))
    display_description = Column (String (255 ))

In fact, the database is defined by one of the fields, there are two main parts: __tablename__ and the definition of the field. This class inherits the two classes of base and novabase, so it's handy to write, Because the Novabase class already provides some basic methods and useful fields (you can view the three classes inherited by Novabase in the nova.openstack.common.db.sqlalchemy.models.py file), such as:

Class Timestampmixin (object):
    created_at = column (DateTime, default=timeutils.utcnow)
    updated_at = column ( DateTime, Onupdate=timeutils.utcnow)

This class provides creation time and update time (note that the fields provided by Novabase also need to be written in the definition of the 133_FOLSOM.PY table structure).

3.api of writing
With model, you need to provide the API to operate this model, into the nova.db.sqlalchemy.api.py.
APIs are generally divided into the creation, query, update, delete, to flavor some operations as an example:
Create:

@require_admin_context def flavor_create (context, values): "" "Create a new instance type. In extra specs, the values dict should contain a ' extra_specs ' key/value pair: {' extra_specs ':

    {' K1 ': ' v1 ', ' K2 ': ' V2 ', ...}}
            "" "Specs = Values.get (' extra_specs ') specs_refs = [] If specs:for K, V in Specs.iteritems (): Specs_ref = models. Instancetypeextraspecs () specs_ref[' key ' = k specs_ref[' value ' = v specs_refs.append (specs_ref) values[' extra_specs '] = specs_refs Instance_type_ref = models. Instancetypes () instance_type_ref.update (values) Try:instance_type_ref.save () except Db_exc. Dbduplicateentry as E:if ' Flavorid ' in E.columns:raise exception. Instancetypeidexists (flavor_id=values[' Flavorid ']) raise exception. Instancetypeexists (name=values[' name ') except Exception as E:raise Db_exc. Dberror (e) return _dict_with_extra_specs (INSTANCE_TYPE_REF) 

@require_admin_context

This is a modifier function, which is equivalent to a function processing, where this modifier is to check the context to see if it is Admin permissions. This can be customized, general use

@require_context

means that you need to pass in the context in order to execute.

Instance_type_ref = models. Instancetypes ()

This creates an instance of the Instancetypes class in models and adds a new record to the instance by Instance_type_ref.update (values) (update is provided by this class of Novabase, Very convenient). The record is then stored in the datasheet via Instance_type_ref.save () (save is also provided by the class Novabase). You can refer to this class in nova.openstack.common.db.sqlalchemy.models.py modelbase.

To create a summary of methods:
1. Need to pass in context and values (this is a dictionary element that contains field and field values)
2. Create an instance of the datasheet model (Somemodel_ref = models. Somemodel ())
3. Update values to the instance (Somemodel_ref.update (values))
4. Deposit in Datasheet (Somemodel_ref.save ())

Inquire:

@require_context def flavor_get (context, ID): "" "Returns a dict describing specific instance_type." "
    result = _instance_type_get_query (context). \ filter_by (id=id). \ a () If not result:raise exception. Instancetypenotfound (Instance_type_id=id) return _dict_with_extra_specs (Result) def _instance_type_get_query ( Context, Session=none, read_deleted=none): query = Model_query (context, models. Instancetypes, Session=session, read_deleted=read_deleted). Options (Joinedl Oad (' extra_specs ') if not context.is_admin:the_filter = [models. Instancetypes.is_public = = True] The_filter.extend ([Models. InstanceTypes.projects.any (project_id=context.project_id)]) query = Query.filter (Or_ (*the_filter)) re Turn Query Def model_query (context, model, *args, **kwargs): "" "the query helper that accounts for the context ' s ' read_de Leted ' Field. :p Aram Context:context to query under:p Aram Session:if present, the session to use:p Aram Read_deleted:if
    ENT, overrides context ' read_deleted field. :p Aram Project_only:if Present and context are user-type, then restrict query to match the context ' s project_i
    D. If set to ' Allow_none ', restriction includes project_id = none. :p Aram Base_model:where Model_query is passed a "model" parameter which are not a subclass of Novabase, we sho  Uld pass a extra Base_model parameter this is a subclass of Novabase and corresponds to the model
    Parameter. "" "Session = Kwargs.get (' session ') or get_session () read_deleted = Kwargs.get (' read_deleted ') or Context.read_del eted project_only = kwargs.get (' project_only ', False) def issubclassof_nova_base (obj): Return isinstance ( obj, type) and Issubclass (obj, models. Novabase) Base_model = model if not issubclassof_nova_base (bAse_model): Base_model = Kwargs.get (' Base_model ', None) if not Issubclassof_nova_base (Base_model):

    Raise Exception (_ ("model or Base_model parameter should be" "subclass of Novabase")) query = Session.query (model, *args) Default_deleted_value = Base_model.__mapper__.c.deleted.default.arg if re ad_deleted = = ' No ': query = query.filter (base_model.deleted = = default_deleted_value) elif read_deleted = ' ye S ': Pass # Omit the filter to include deleted and active elif read_deleted = = ' only ': query = QUERY.F Ilter (base_model.deleted!= default_deleted_value) Else:raise Exception (_ ("Unrecognized read_deleted value"% S ' ")% read_deleted) if Nova.context.is_user_context (context) and project_only:i F project_only = = ' Allow_none ': query = query.\ filter (or_ (base_model.project_id = = CONTEXT.PR
              OJECT_ID,             base_model.project_id = None) Else:query = query.filter_by (project_id=context.project_
     ID) return Query

Model_query is the function that actually executes the query, which is to get a session of the database first (this is an instance of the SQLAlchemy definition, You can view the Get_session function in nova.openstack.common.db.sqlalchemy.session.py and return a query (this is through the session instance's Query method, The returned data obtained after querying the data table based on the parameters passed in.

You don't have to care about this when you're writing the query API. The main is the writing of this function:

def _instance_type_get_query (context, Session=none, Read_deleted=none):
    query = model_query (context, models. Instancetypes, Session=session,
                       read_deleted=read_deleted). \
                       Options (joinedload (' extra_specs '))
    if Not context.is_admin:
        the_filter = [Models. Instancetypes.is_public = = True]
        the_filter.extend ([
            models. InstanceTypes.projects.any (project_id=context.project_id)
        ])
        query = Query.filter (Or_ (*the_filter))
    return query    

Model_query need to pass in the context, model (the name of the models), session, and some parameters such as read_deleted (read_deleted value is "yes" or "no", Used to choose whether or not to get deleted as true, because OpenStack rarely deletes records, but changes the deleted value of records from 0 to 1.
A few complete examples:
Gets the first data that has not been deleted in the Somemodel ID field value of some_id:

result = Model_query (context, models. Somemodel, session=session,
                     read_deleted= "no"). \
            filter_by (id=some_id). \ a
            ()

Gets the first data that has not been deleted in the Somemodel with the Size field value of Some_size:

result = Model_query (context, models. Somemodel, session=session,
                     read_deleted= "no"). \
            filter_by (size=some_size). \ a
            ()       

Gets all data that has not been deleted in the Somemodel ID field value of some_id:

result = Model_query (context, models. Somemodel, session=session,
                     read_deleted= "no"). \
            filter_by (id=some_id). \ all
            ()

That's probably the way it is.
Summary of query methods:
1. Write a correct query statement according to Model_query
2. Create filter or do not create
3. Select All or the one (in fact, the difference between the list and the string)

Update:

@require_context
def some_update (context, some_id, values): Sessions
    = Get_session () with
    Session.begin () :
        Some_ref = some_get (Context, some_id, session=session)
        some_ref.update (values)
        Some_ref.save (session =session)

Update this is simpler, and very similar to the creation, the difference is that the some_ref in the creation method is an instance of the data model in models.py, and the Some_ref in the Update method is a Model_query instance obtained through the query.

This method is not summed up.

Delete:

@require_admin_context
def some_destroy (context, some_id): Sessions
    = Get_session () with
    Session.begin () :
        Session.query (models. Somemodel). \
                filter_by (id=some_id). \
                Update ({' deleted ': True,
                        ' deleted_at ': Timeutils.utcnow (),
                        ' Updated_at ': Literal_column (' Updated_at ')})

Deleting this is also easier to understand, query to the record that needs to be deleted, and update deleted to True (or 1, and so on) with a Boolean value of true.

This is not summed up.

4. Encapsulate the API
Enter nova.db.api.py
Add the API written in nova.db.sqlalchemy.api.py to this file, example:

def some_create (context, values): Return
    impl.some_create (context, values)

def some_destroy (context, some_id ): Return
    Impl.some_destroy (context, some_id)

def some_get (context, some_id): Return
    Impl.some_get ( Context, some_id

def some_get_by_size (context, Some_size): "Return Impl.some_get_by_size" (Context,
    some_size

def some_get_all (context): Return
    impl.some_get_all

def some_update (context, some_id, values ): Return
    to Impl.some_update (context, some_id, values)

Here, the programming in Nova.db is basically over.


Second, the compute module in the call method writing
First, the method of calling the database in the e\f version of nova.compute.manager.py:
You can import DB through the manager class, and it is very convenient to use SELF.DB.SOME_API to directly drop the Nova.db.api method t_t

Again g\h version of the conductor to invoke the nova.db:
When you want to invoke the method in Nova.db.api, you need to add the appropriate method to the following files.
example, now there is a method in Nova.db.api: some_create (context, values) is used to create a record, A feature in nova.compute.manager.py will use it, and there must be a statement in this function:

Self.conductor_api.some_create (Ontext, values)

(Of course in this case, either the function name or the parameter can be arbitrary, in addition to the context)
It will be transferred into the some_create of the Localapi class in nova.conductor.api.py (Localapi represents the operation of this function at this compute node) Localapi method can be written in this way:

def some_create (self, context, values): Return
    self._manager.some_create (context, values)

Then the Some_create method of the Conductorapi class in nova.conductor.rpcapi.py can be written:

def some_create (self, context, values):
    cctxt = self.client.prepare (version= ' 1.50 ') return
    Cctxt.call ( Context, ' some_create ',
                        values=values)

Here conductor also calls RPC, which means that the action recorded in the Create data table is sent back to the control node by the message, and the control node is assigned to the some_ of the Conductormanager class in the nova.conductor.manager.py. Create method to execute.
In this method, you can do the same as the e\f version of the nova.compute.manager.py the same operation, is directly called the nova.db.api.py method.

def some_create (self, context, values):
    self.db.some_create (context, values)

This basically completes a function in the operation of the Nova database programming.