Registry v2 parsing and how token verification is implemented

This is a creation in Article, where the information may have evolved or changed.

Referring to registry v2, the main improvement is to support the parallel pull image, the mirror layer ID becomes unique, solve the problem that the same tag may correspond to multiple mirrors, and so on. If you don't know enough, you can listen to me carefully.

The first thing to say is that V2 has added a new concept digest

He is a string of hash values calculated from the content-based addressing (content-addressable) algorithm. Simply said that the content is different, the digest value is different, but the content is the same, the resulting digest value is necessarily the same. Each of our mirror layer IDs is digest based on the content of each mirrored layer.

So you change the mirror layer after the generation of digest is different, but not moving, the digest is still unchanged, then this digest ID when generated? The mirror layer ID generated when we built the image locally is different every time, and this digest is generated when we push the image.

To verify that the content is the same, push to registry get the same digest, I did a little experiment, with the following dockerfile comment off the third line and not comment on the construction of two mirrors, and then push to registry

If it is V1, the layer ID of the push will certainly be different, but inside the V2, the third line of the note gets 5 mirror layers, without commenting out the third line to get 6 mirror layers, and the first 5 layers are included in the second 6. So it is concluded that this digest is actually generated from the content.

Next, the ID of the mirror.

The image ID also generates a digest value, which is the digest generated from some other information, such as the _manifest file, which is the mirror layer ID and the mirror name. Each time we push the mirror to V2, we end up returning a digest value to the Docker client, which represents the digest ID of the mirror. The purpose of this ID is to specify a unique image. Similar to tag use.

Because we know V1 time with tag has a disadvantage is that the image can use the same tag multiple times, causing us to use tag to identify the image may not be what we want, and the use of digest will not occur.

When we write dockerfile, we can use this as a reference to the image:

From localhost:5000/test@sha256:ac81211548c0d228e10daaf76f6e0024e5f91879c8a7e105e777d6f964270449

As with tag, you can use the Docker images--digests when viewing the image digests with local Docker:

Now, of course, all you see is <none> we need to pull down from registry and use
Docker Pull localhost:5000/test@sha256:ac81211548c0d228e10daaf76f6e0024e5f91879c8a7e105e777d6f964270449

To understand digest, let's take a look at the storage structure of registry

This section is best to look at the folder structure of registry. A simple sketch is drawn.

is a folder-level relationship V2

This is the directory under the specific appearance, you can see for a moment, you can look at the registry below there are two folders BLOBs and repositories,

BLOBs below stores all the basic information elements of registry, including the Mirror Layer Digest and Mirror Digest, after which the information is called here in some way.

BLOBs folder first divided into a level, is the digest of the first two characters of the folder in order to filter the digest, to avoid this folder is too large, view is difficult. This facilitates positioning. Each blob has a data file that stores relevant information.

Repositories is stored under each image name of the folder, enter a mirror folder, you can see three subfolders, _layers, _manifests, _uploads,_layers This folder is related to this image of all the mirror layer, into one of the mirror layer folders, the following is only one file--link named file, inside the content is a digest.

This is connected to the mirror layer below the blobs, and in the Repositories folder, it is connected through the link file with the file under the BLOBs folder.

_upload This folder, usually point in is empty, this folder is the main role of uploading. When we upload the mirror layer, first upload to this folder, and so on after the transfer, after the contents of this folder moved to BLOBs, and then the original file deleted.

_manifest This folder is very important, is the information about the image. He has two sub-folders below, revisions and tags,revisions This folder is the image of all the available image digest, the link file inside the image of the digest. We go to blobs inside to find the corresponding ID corresponding to the file, view the file below the data, we found that the data file stored in the information, and we registry through the V2 REST API request manifest information is the same ~ in See _manifest/tags/. Here is a different tag for this image. It also points out that current and index represent the digest and all Mirror Digest under this tag, respectively.

Here's a look at how to build token verification registry

Let's take a look at the official map.

As you can see, V2 and V1 have completely changed the form of access, removing the index server and adding a auth server.

The order of access is this

1. We let Docker Deamon access registry,registry through the Docker client. If no authentication is required, return the result directly, and if required, return 401 and include some information in the header,

2. Daemon access to Auth server based on information. Auth server judged passed the validation and returned to daemon a bunch of tokens.

3. Daemon with this token to visit registry can get information, pull, push,api the process of walking.

Next I paste my profile config.yml, configure the option Auth/token to indicate token verification. This process really needs a good read, and it has a lot to do with her encryption.

version:0.1
Log
Fields
Service:registry
Storage
Cache
Blobdescriptor:redis
FileSystem
RootDirectory:/var/lib/registry
http
Addr:: 5000
Secret:randomstringsecret
Tls:
Certificate:/ROOT/SSLKEYS/DOMAIN.CRT
Key:/root/sslkeys/domain.key
Auth
Token
Realm:https://registry.tenxcloud.com:5001/auth
service:test123.tenxcloud.com:5000
Issuer:qwertyui
Rootcertbundle:/ROOT/SSLKEYS/DOMAIN.CRT
Of course, I was V2 through the container, and I put this config volume in.

My startup command: Docker run-d-P 5000:5000--restart=always--name registry-v pwd /sslkeys:/root/sslkeys-v pwd /config.yml:/et C/docker/registry/config.yml-v pwd /data:/var/lib/registry registry:2.1.1

The 4 sub-options under Auth/token must be configured, realm represents my auth server address, service represents the registry address, issuer is a string of identifiers, just write it, auth The same string needs to be configured when the server is encrypted. The rootcertbundle configures a secret key to encrypt tokens. (I've reused my TLS token here.)

When we first visit registry without token, we return 401 with the following information: Www-authenticate:bearer realm= "test123.tenxcloud.com:5000", service= " test123.tenxcloud.com:5000 "scope=" Repository:husseingalal/hello:push ", realm and service is what I said earlier, scope represents the action I want to do, The repository represents the image, followed by the mirror name, then pull or push, or both

Knowing the function of each parameter, we can build our auth server, I found a project from GitHub: Https://github.com/cesanta/docker_auth

Written in the go language, where the account password is stored: Write files, LDAP and Google account. This server implements the dynamic load configuration file, the configuration file changes this server will be a safe restart, so you can dynamically add account password to the file. Of course, you can also write your own authentication, add a database, and so on, just inherit authenticator this interface can. Add it up easily. After authentication has permission control ACL, and we can also customize permission control, inherit Authorizer this interface can.

The first two pass is to generate tokens (this part of the project has been packaged, without modification),
V2 token uses the JWT encryption method, JWT divided three parts, Header,payload,signature,header inside with the information is token encryption, generally base64, payload brought is the basic information needed, User, permission, expiration time, and issure, and so on.
Signature is header+payload after the secret key to encrypt, this key is configured in registry Rootcertbundle corresponding key.

Three parts passed. Connection, because there is secret key encryption, to ensure that token information will not be tampered with, this encryption method to ensure that the authorization and registry separation is also secure, send token back to daemon daemon can take token to normal access registry, If it's a rest API, just write it bearer token and you can request it.

This article author: Speed Cloud engineer Ding Qiwei Source Link: Registry v2 analysis and how to implement token verification