Explore OpenStack (10): Deep mirror Service glance

This blog post to explore the next image service glance.

0. Basic concept 0.1 Basic functions

Glance provides the rest API to support the following mirroring operations:

• Inquire
• Registered
• Upload
• Get
• Delete
• Access Rights Management

0.2 Glance REST API version V1 and V20.2.1 feature differences

Glance has two versions of rest API V1 and V2, and there's a pretty big difference between the two:

(1). V1 provides only basic image and member operation features: image creation, deletion, download, list, detail query, update, and creation, deletion, and list of mirror tenant members.

(2). In addition to supporting all functions of the V1, V2 mainly adds the following functions:

• Image location additions, deletions, and modifications
• Metadata namespace operation
• Image tag Operation

(3). V1 and V2 support for the image store is the same.

0.2.2 Implementation Differences

V1 implementation, there are GLANCE-API and glance-registry two WSGI services, all provide rest API, but Glance-api REST API for external use, The Glance-registry API is used only by GLANCE-API.

and V2 in the realization, the function of Glance-registry merged into the GLANCE-API, reducing an intermediate link.

0.2.3 Use Difference

Currently, the Glance CLI and horizon are using the V1 version of the API by default. The reason,

• On the one hand, it seems that the internet has seen that V2 performance is not V1 good, we can also see the use of V2 API, the need to call the rest API multiple times;
• On the other hand, I think the newly added features in V2 do not seem to be the necessary function of image management, so it is reasonable to use them temporarily.

Data storage for 0.3 image

The metadata for image is stored in the DB via Glance-registry, and the chunk data of the image is stored in various backend stores through glance-store and obtained from it.

0.4 access rights for image

The access rights of image are divided into:

• Public: Can be used by all tenant.
• Private/Project: can only be used by the tenant of image owner.
• Shared: A non-shared image can be shared with another tenant, and can be implemented by member-* operations.
• Protected protected: The image of protected cannot be deleted.

0.5 the various states of the image

• Queued: Image data is not uploaded, only metadata in db.
• Saving: Uploading image data
• Active: normal state
• Deleted/pending_delete: deleted/Pending removal
• Killed:image metadata is incorrect and waits to be deleted.

State diagram:

• ' Queued ' = (' saving ', ' active ', ' deleted ')
  ' Saving ' = (' active ', ' killed ', ' deleted ', ' queued ')
  ' Active ' = (' queued ', ' pending_delete ', ' deleted ')
  ' Killed ' = (' deleted ')
  ' Pending_delete ' = (' deleted ')
  ' Deleted ' = ()

1. V1 version of code implementation 1.1 code module

As can be seen, Glance V1 is mainly three modules:

• glance/common/wsgi.py-WSGI Server, open glance-(api|registry) service, accept requests, such as get/images/ Bb8838d5-06b5-4f7e-b6ef-87c908f04cc7 http/1.1
• GLANCE-API: Provides external rest API, calls Glance-registry for database operations, and calls Glance-store for store operations
  • Images.py-Implements the image-* operation Memebers.py-implements the member-* operation router.py-Maps the HTTP operation received by WSGI to the Controller class of the Glance-api module.
• glance-registry: Providing metadata database operations to GLANCE-API REST API, which is monitored by default on port 9191. Main documents:
  • Client/v1/api.py--Registry API entry, calling function client.py in client.py--calls images.py in members.py and functionapi/v1/ images.py--Call glance/db/sqlalchemy/api.py to implement images database related operations memebers.py--Call glance/db/sqlalchemy/api.py Implementing the related operations of the Members database table glance/db/sqlalchemy/api.py--DB Operation API entry models.py--database table model, one class per table

• Glance-store: This part of the code is not in the Glance GitHub code project and doesn't understand why it's not together, Cinder is a similar architecture, but all cinder's code is in a GitHub project.
  • Glance-store provides file backend.py to  glance-api  as a unified portal for store operations. The main functions include:
    • create_stores: Call the function to register each store
    • get_from_ when the Store.configure_add,glance-api service is started, based on user Configuration Backend: Call Store.get, get data chunks
    • get_size_from_backend from store: Call store.get_size, get image size from store
    • delete_from_backend: Call Store.delete to remove image data from store
    • store_add_to_backend: Call Store.add, add image to store
    • set_acls: Call Store.set_acls to set read and write permissions for image
  • The
  • Glance-store provides a base class store that requires the implementation of each store vendor, and defines several actions that need to be implemented:
    • configure_ Add: Configure the Backend store
    • get: Get image chunk data
    • get_size: Gets the image size
    • add: Add image to store
    • delete: Remove image chunk data from store
    • SET_ACLS: Set Read and Write permissions for image
  • currently supported by the backend store see. The features supported by the backend store vary widely, such as
    • cinder.py only implements the method  get_size, because the current Glance does not support upload image   to Cinder volume.
    • http.py implements only get and get_size, so only the HTTP store download image and query image s Ize
    • swift.py's implementation can be said to be the most complete, filesystem second.
  • Using the backend Store
    • The store that is used by default is specified by the configuration item default_store=<store> configuration file glance-api.conf
    • configuration items that can be used with the stores configuration file glance-api.conf stores=<stores> Specify
    • You can use the parameter--store to specify the target backend store in the glance Cli image-create

1.2 Operation Flow

Glance Cli	Rest API called by the Glanceclient SDK	Glance intrinsic functions	Concrete implementation of Gance internal function
Image-create Create image	Post/images	glance/api/v1/images.py def create (self, req, Image_meta, Image_data): Adds a new image to Glance	1. api/v1/images.py: _reserve. Add image metadata to db, get the image ID, and set the image status to queued. 1.1. Registry/client/v1/client.py:def Add_image (self, image_metadata) 1.2. Registry/api/v1/images.py:def Create (self, req, body) 1.3. Db/sqlalchemy/api.py:image_create 2. api/v1/images.py: _handle_source Image data is processed in three cases based on the image data source: 2.1 If Image_data//If you pass in image data directly 2.1.1 Def _upload (self, req, Image_meta): The image data is stored in the specified backend store. Note that the Content-type must be application/octet-stream. 2.1.1.1 Registry.update_image_metadata//Set image status to saving 2.1.1.2 Upload_utils.upload_data_to_store// Check_quota//Check user quota Store_api.store_add_to_backend//method of calling Backend.py Store.add//Calling the backend Store implementation method filesystems.py. def Add//If it is filesystem, write image chunk data to a file with the image ID as the filename 2.1.2 def _activate//Call Glance-registry set image status to Active 2.2 If it is copy_from, asynchronously executes the above 2.1.1 ~ 2.1.2 2.3 If it is a designated location, 2.3.1 Gets the image size from the backend store specified in location 2.3.2 Setting the image status to active
Image-update Modify the image metadata property or data	Put/images/<id>	glance/api/v1/images.py def update (self, req, id, Image_meta, image_data) Updates an existing image with the registry	Attention: (1) Only use location| allowed Copy-from to modify an image with a status of queued (2) You can modify the image Access property, as well as the metadata. 1. Modify the image access permission, call Registry.get_image_members, and then call the Store.set_acls method, but it only has swift support. 2. Modify image metadata to call the Registry.update_image_metadata method 3. Upload image data, call Self._handle_source, see above #2.
Image-delete Deletes the specified image based on the image ID	Delete/images/<id>	glance/api/v1/images.py def delete (self, req, id) Deletes the image and all it chunks from the Glance	1. Call Registry.get_image_metadata to get image metadata 2. Check whether the image can be deleted, such as whether it is protected, etc. 3. Call Registry.update_image_metadata to set the status to deleted or Pending_delete 4. Call Upload_utils.initiate_deletion to start the delete operation 4.1 calls the Delete_from_backend method of backend.py, which invokes Store.delete, and calls the Os.unlink (FN) method to delete the file if the image is stored in the file system. 4.2 Db_api.get_api (). Image_location_delete//Set the ID of the table image_locations and image_id corresponding to item deleted,status,updated_at, Deleted_at 5. Call Registry.delete_image_metadata to delete image metadata 5.1 Registry/images.py:def Delete (self, req, id) 5.1.1 db/sqlalchemy/api.py: _image_locations_delete_all//delete All image entries in DB table image_locations for the GIV En image ID 5.1.2 db/sqlalchemy/api.py: _image_child_entry_delete_all//deletes All the entries in DB table image_properties for the Given image ID 5.1.3 db/sqlalchemy/api.py: _image_child_entry_delete_all//deletes All the entries in DB table image_members for the Give N Image ID 5.1.4 db/sqlalchemy/api.py: _image_child_entry_delete_all//deletes All the entries in DB table Image_tags for The given image ID
Image-list Get image List	Get/images--Returns a set of brief metadata about images	glance/api/v1/images.py def index (self, req) Returns The following information for all public, available images: ID--The opaque image identifier Name-the name of the image Disk_format-the disk image format Container_format-the "container" format of the image Checksum--MD5 checksum of the image data Size--size of image data in bytes Like what: { "Name": "Name3", "Container_format": null, "Disk_format": null, "Checksum": null, "id": "1e7b0a5a-7b78-4673-8d56-3abff7b491ae", "Size": 12336128 }	Call registry.get_images_list to get image metadata from DB
Get more information about all the image	Get/images/detail--Returns a set of detailed metadata about images	glance/api/v1/images.py def detail (self, req) Returns detailed information for all available images. Like what: { "Status": "Queued", "Deleted_at": null, "Name": "Name3", "deleted": false, "Container_format": null, "Created_at": "2015-01-21t23:05:54", "Disk_format": null, "Updated_at": "2015-01-21t23:05:54", "Min_disk": 0, "Protected": false, "id": "1e7b0a5a-7b78-4673-8d56-3abff7b491ae", "Min_ram": 0, "Checksum": null, "Owner": "Fa2046aaead44a698de8268f94759fc1", "Is_public": false, "Virtual_size": null, "Properties": {}, "Size": 12336128 },	Gets the data for the image directly from the DB. 1. Call Registry.get_images_detail 1.1 Calling the Def detail in registry/api/v1/images.py 1.1.1 calls the Image_get_all in db/sqlalchemy/api.py
Glance image-show Displays metadata for the specified image based on the image ID	head/images/ <ID>-Return metadata about a image with ID <id>	glance/api/v1/images.py  de F Meta (self, req, id) Returns metadata about a image in the HTTP headers of the response object	Get image partial metadata from DB 1. Call self.get_image_meta_or_404 (self, request, image_id) 1.1 call   Registry.get_image_metadata 1.1.1  call  db_api.image_get
image-download   Download image	get/images/<id > --Return image data for image with ID <id>	GLANCE/API/V1/IMAGES.PY&NBSP; Def SH ow (self, req, id) Returns A iterator that can is used to retrieve a image ' S  data along with the IMA GE metadata	egistry.get_image_metadata, and then call   self.db_ Api.image_get to get the image metadata  2. Call Self._get_from_store to get the image data  2.1 call Glance_store.location.get_location_from_uri, get the location2.2 from the image URI Using the backend.py Get_store_from_uri, get the store2.3 call Store.get get the image data
Member-list--image-id Gets a list of tenant members that can access the specified image	get  /images/{image_id}/members	GLANCE/API/V1/MEMBERS.PY&NBSP; Def index (self, req, image_id) Return A list of dictionaries indicating the members of The  image, i.e., those tenants the image is Shared with	performs database operations through Glance-registry. 1.1 registry.get_image_members 1.1.1 registry/api/v1/members.py: def index 1.1.1.1db/sqlalchemy/api.py: image_get 1.1.1.2db/sqlalchemy/api.py: image_member_find
Member-list--tenant-id Gets the list of accessible image for the specified tenant	get / Shared-images/{id}	GLANCE/API/V1/MEMBERS.PY&NBSP; def index_shared_images (self, req, id) Retrieves list of image memberships for the given member.	Performs database operations through Glance-registry. 1.1 registry.get_member_images 1.1.1 registry/api/v1/members.py: def index_shared_ Images 1.1.1.1 db/sqlalchemy/api.py: image_member_find
Member-create Adds the specified tenant accessible member to the specified image	put /images/ {Image_id}/members/{id}	glance/api/v1/members.py def update (self, req, image_id, ID, body =none) Adds A membership to the image, or updates an existing one.	Performs database operations through Glance-registry. 1.1 registry.add_member 1.1.1 registry/api/v1/members.py:def Delete 1.1.1.1 db/sqlalchemy/ Api.py: image_update
Member-delete Deletes the specified accessible tenant member of the specified image	Delete/images/{image_id}/members/{id}	glance/api/v1/members.py def delete (self, req, image_id, id)//removes a membership from the image	Perform database operations through Glance-registry. 1.1 Registry.delete_member 1.1.1 Registry/api/v1/members.py:def Delete 1.1.1.1db/sqlalchemy/api.py:image_get 1.1.1.2 Db/sqlalchemy/api.py:image_member_find 1.1.1.3 Db/sqlalchemy/api.py:image_member_delete

2 Summary

The principle and implementation of Glance is simpler and more straightforward than other components. V2 REST API content has increased a lot, and further research is needed.

Explore OpenStack (10): Deep mirror Service glance