Deploying Private Docker Registry

This is a creation in Article, where the information may have evolved or changed.

Installing and deploying a private Docker registry is one of the only necessary technologies for introducing, learning and using Docker. Especially when Docker is accepted by the organization and more people, projects and products begin to touch and use Docker, storing and distributing the homemade Docker image is just as needed. Docker registry, as always, inherits the characteristics of "Docker pit many", so here will build their own "various" registry process of implementation of the steps, encountered problems recorded, for their own forgotten, for his reference.

Docker launched the distribution project in 2015, Docker Registry 2. There are significant improvements in security and performance compared to the old Registry,registry 2 using GO implementations. Registry has designed a new rest API and is no longer compatible with old Registry in the image storage format. Last August, the Docker official hub used Registriy 2.1 instead of the original old Registry. If you want to interact with Registry2, your Docker version is at least Docker 1.6.

Docker developers have also been working to improve the experience of registry installation and use, simplifying registry configuration by providing official registry image and Docker compose tools. In this article, however, we just use Docker and the official image of registry to deploy registry, which makes it easier to fully understand the deployment configuration details of registry.

Registry2 not only supports this site in terms of mirrored storage, but also supports many mainstream third-party storage solutions. With distributed storage systems you can also implement a distributed Docker Registry service. This is the case only for this site and single node Registry2.

First, the environment

Here is also the reuse of Docker environments in previous articles:

Docker Registry Server: 10.10.105.71 Ubuntu 14.04 3.16.0-57-generic; docker 1.9.1

Two other working servers:
10.10.105.72 Ubuntu 14.04 3.19.0-25-generic; docker 1.9.1
10.10.126.101 Ubuntu 12.04 3.16.7-013607-generic; docker 1.9.1
This time the Registry uses the latest stable version: Registry 2.3.0. Since the image is stored on a local disk, the root partition is small, and other volumes need to be mapped.

Second, the first build
I thought how easy it is to build a Docker Registry, even as simple as a single command. For example, we execute on Registry Server:

Under ~ / dockerregistry, execute:

$ sudo docker run -d -p 5000: 5000 -v `pwd` / data: / var / lib / registry --restart = always --name registry registry: 2
Unable to find image 'registry: 2' locally
2: Pulling from library / registry
f32095d4ba8a: Pull complete
9b607719a62a: Pull complete
973de4038269: Pull complete
2867140211c1: Pull complete
8da16446f5ca: Pull complete
fd8c38b8b68d: Pull complete
136640b01f02: Pull complete
e039ba1c0008: Pull complete
c457c689c328: Pull complete
Digest: sha256: 339d702cf9a4b0aa665269cc36255ee7ce424412d56bee9ad8a247afe8c49ef1
Status: Downloaded newer image for registry: 2
e9088ef901cb00546c59f89defa4625230f4b36b0a44b3713f38ab3d2a5a2b44

$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
registry 2 c457c689c328 9 days ago 165.7 MB

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e9088ef901cb registry: 2 "/ bin / registry / etc / d" About a minute ago Up About a minute 0.0.0.0:5000->5000/tcp registry
The Registry container has run up, and its startup log can be viewed through: docker logs registry.

We tag busybox: latest locally at 71 and try to push the image under the new tag to the Registry:

$ docker tag busybox: latest 10.10.105.71:5000/tonybai/busybox:latest
$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
registry 2 c457c689c328 9 days ago 165.7 MB
busybox latest 65e4158d9625 9 days ago 1.114 MB
10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB
...

Push to the Registry:

$ docker push 10.10.105.71:5000/tonybai/busybox
The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1)
unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: Tunnel or SSL Forbidden
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: Tunnel or SSL Forbidden
error! After a brief analysis, it may be that the http proxy is added to the docker daemon configuration on 71, which makes it impossible to ping the registry endpoint. So comment out the export http_proxy = ”xxx” setting in / etc / default / docker and restart the docker daemon.

Try push again:

$ docker push 10.10.105.71:5000/tonybai/busybox
The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1)
unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527
Although it still fails, the error message is different. This time it seems that the connection can be established, but the client side accesses the server side through https, it seems to want to tls communication, but this process has not been completed.

Trying to push image to registry on other machines also encountered the same error output, as follows:

10.10.105.72:

$ docker push 10.10.105.71:5000/tonybai/ubuntu
The push refers to a repository [10.10.105.71:5000/tonybai/ubuntu] (len: 1)
unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527
Judging from the error message, the client interacts with the Registry and will use https access by default, but we did not configure any key and crt files related to tls when installing Registry. Https access must fail. To understand this problem, you can only check the Registry Manual.

3. Insecure Registry
The documentation of Registry is still relatively detailed. In the documentation, we found the Insecure Registry, the configuration and usage of the Registry that receives plain http access, although this is not officially recommended.

In fact, for our internal network, Insecure Registry can basically meet the needs, and the deployment process also avoids the cumbersome steps of the Secure Registry, such as making and deploying certificates.

In order to build an Insecure Registry, we need to first clean up the Registry container that has been started above.

$ docker stop registry
registry
$ docker rm registry
registry
Modify the configuration of the Docker daemon on the Registry server to add –insecure-registry to DOCKER_OPTS:

DOCKER_OPTS = "-insecure-registry 10.10.105.71:5000 ....
Restart the Docker Daemon and start the Registry container:

$ sudo service docker restart
docker stop / waiting
docker start / running, process 6712
$ sudo docker run -d -p 5000: 5000 -v `pwd` / data: / var / lib / registry --restart = always --name registry registry: 2
5966e92fce9c34705050e19368d19574e021a272ede1575385ef35ecf5cea019
Try to push image again:

$ docker push 10.10.105.71:5000/tonybai/busybox
The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1)
65e4158d9625: Pushed
5506dda26018: Pushed
latest: digest: sha256: 800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739
This time push ok!

We will untag the local tag, and then pull the relevant image from the Registry:

$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
registry 2 c457c689c328 9 days ago 165.7 MB
10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB
busybox latest 65e4158d9625 9 days ago 1.114 MB
ubuntu 14.04 6cc0fc2a5ee3 5 weeks ago 187.9 MB

$ docker rmi 10.10.105.71:5000/tonybai/busybox
Untagged: 10.10.105.71:5000/tonybai/busybox:latest

$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
registry 2 c457c689c328 9 days ago 165.7 MB
busybox latest 65e4158d9625 9 days ago 1.114 MB
ubuntu 14.04 6cc0fc2a5ee3 5 weeks ago 187.9 MB

$ docker pull 10.10.105.71:5000/tonybai/busybox
Using default tag: latest
latest: Pulling from tonybai / busybox
Digest: sha256: 800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892
Status: Downloaded newer image for 10.10.105.71:5000/tonybai/busybox:latest

$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
registry 2 c457c689c328 9 days ago 165.7 MB
10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB
busybox latest 65e4158d9625 9 days ago 1.114 MB
ubuntu 14.04 6cc0fc2a5ee3 5 weeks ago 187.9 MB
As you can see, the Pull process is also very smooth.

View or retrieve the Repository or images in Private Registry2, you will not be able to use docker search:

$ docker search 10.10.105.71:5000/tonybai/busybox/
Error response from daemon: Unexpected status code 404
But through the v2 version of the API, we can achieve the same purpose:

$ curl http://10.10.105.71:5000/v2/_catalog
{"repositories": ["tonybai / busybox"]}

$ curl http://10.10.105.71:5000/v2/tonybai/busybox/tags/list
{"name": "tonybai / busybox", "tags": ["latest"]}
On other hosts, we try to pull busybox:

10.10.105.72:

$ docker pull 10.10.105.71:5000/tonybai/busybox
Using default tag: latest
Error response from daemon: unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527

We found that it is still impossible to pull and push! As mentioned in the Registry manual, if you use the insecure registry mode, then the Docker Daemon on all hosts that interact with the Registry must be configured with the –insecure-registry option.

We modify the / etc / default / docker on 105.72 according to the above configuration method, restart the Docker daemon, and then execute pull / push to get the correct result:

$ sudo vi / etc / default / docker
$ sudo service docker restart
docker stop / waiting
docker start / running, process 10614
$ docker pull 10.10.105.71:5000/tonybai/busybox
Using default tag: latest
latest: Pulling from tonybai / busybox
5506dda26018: Pull complete
65e4158d9625: Pull complete
Digest: sha256: 800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892
Status: Downloaded newer image for 10.10.105.71:5000/tonybai/busybox:latest

$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
ubuntu 14.04 36248ae4a9ac 8 days ago 187.9 MB
10.10.105.71:5000/tonybai/ubuntu 14.04 36248ae4a9ac 8 days ago 187.9 MB
10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB

$ docker push 10.10.105.71:5000/tonybai/ubuntu
The push refers to a repository [10.10.105.71:5000/tonybai/ubuntu] (len: 1)
36248ae4a9ac: Pushed
8ea5373bf5a6: Pushed
2e0188208e83: Pushed
e3c70beaa378: Pushed
14.04: digest: sha256: 72e56686cb9fb38438f0fd68fecf02ef592ce2ef7069bbf97802d959d568c5cc size: 6781
4. Secure Registry
Docker official recommends that you use the Secure Registry working mode, that is, transport uses tls. So we need to configure the key and crt files required by tls for Registry.

We first clean up the environment, stop the above Insecure Registry and rm off; remove --insecure-registry in the DOCKER_OPTS configuration of the Docker Daemon on each host, and restart the Docker Daemon.

If you own a domain name, the host under the domain name provides Registry services, and you have a certificate file signed by a well-known CA, then you can build a Secure Registry. However, I do not have a ready-made certificate here, so I can only use a self-signed certificate. Strictly speaking, the use of self-signed certificates still belongs to Insecure in the eyes of Docker officials, but here is just to use self-signed certificates to explain the deployment steps of Secure Registry.

1. Make a self-signed certificate
If you have a certificate signed by a well-known CA, then this step can be ignored directly.

$ openssl req -newkey rsa: 2048 -nodes -sha256 -keyout certs / domain.key -x509 -days 365 -out certs / domain.crt
Generating a 2048 bit RSA private key
.............. +++
...................................... +++
writing new private key to 'certs / domain.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', The field will be left blank.
-----
Country Name (2 letter code) [AU]: CN
State or Province Name (full name) [Some-State]: Liaoning
Locality Name (eg, city) []: shenyang
Organization Name (eg, company) [Internet Widgits Pty Ltd]: foo
Organizational Unit Name (eg, section) []: bar
Common Name (e.g. server FQDN or YOUR name) []: mydockerhub.com
Email Address []: bigwhite.cn@gmail.com
2. Start Secure Registry
Start Registry with certificate:

$ docker run -d -p 5000: 5000 --restart = always --name registry \
  -v `pwd` / data: / var / lib / registry \
  -v `pwd` / certs: / certs \
  -e REGISTRY_HTTP_TLS_CERTIFICATE = / certs / domain.crt \
  -e REGISTRY_HTTP_TLS_KEY = / certs / domain.key \
  registry: 2
35e8ce77dd455f2bd50854e4581cd52be8a137f4aaea717239b6d676c5ea5777

Since the CN of the certificate is mydockerhub.com, we need to modify the / etc / hosts file:

10.10.105.71 mydockerhub.com
Recreate a tag for busybox:

$ docker tag busybox: latest mydockerhub.com:5000/tonybai/busybox:latest
Push to Registry:

$ docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
unable to ping registry endpoint https://mydockerhub.com:5000/v0/
v2 ping attempt failed with error: Get https://mydockerhub.com:5000/v2/: x509: certificate signed by unknown authority
 v1 ping attempt failed with error: Get https://mydockerhub.com:5000/v1/_ping: x509: certificate signed by unknown authority
push failed! From the error log, the docker client believes that the signer of the certificate transmitted by the server is an unknown authority (unknown CA), so the verification fails. We need to let the docker client install our CA certificate:

$ sudo mkdir -p /etc/docker/certs.d/mydockerhub.com:5000
$ sudo cp certs / domain.crt /etc/docker/certs.d/mydockerhub.com:5000/ca.crt
$ sudo service docker restart // After installing the certificate, restart Docker Daemon
Execute Push again, and we see the successful output log. Since the tonybai / busybox repository has been previously pushed under the data directory, the prompt "already exists":

$ docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
65e4158d9625: Image already exists
5506dda26018: Image already exists
latest: digest: sha256: 800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739
3. External access to Registry
We try to access this secure registry with another machine. According to the previous requirements, we modify the hosts file according to the cat, install ca.cert, remove the –insecure-registry option, and restart the Docker daemon. Then try to pull the image from the registry:

$ docker pull mydockerhub.com:5000/tonybai/busybox
Using default tag: latest
latest: Pulling from tonybai / busybox

Digest: sha256: 800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892
Status: Downloaded newer image for mydockerhub.com:5000/tonybai/busybox:latest

$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
10.10.105.71:5000/tonybai/ubuntu 14.04 36248ae4a9ac 9 days ago 187.9 MB
ubuntu 14.04 36248ae4a9ac 9 days ago 187.9 MB
10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB
mydockerhub.com:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB
In this way, if you use a self-signed certificate, all Docker hosts that want to interact with the Registry need to install the ca.crt (domain.crt) of mydockerhub.com. But if you use a well-known CA, this step can be ignored.

V. Registry authentication management
Registry provides a basic authentication method. We can add basic authentication to Registry through the following steps:

On the Register server, add the foo user to the Registry and the password foo123: (You need to stop the existing Registry before and delete it)

// Generate authentication password file
$ mkdir auth
$ docker run --entrypoint htpasswd registry: 2 -Bbn foo foo123> auth / htpasswd
$ ls auth
htpasswd

// Start Registry with authentication function:
$ docker run -d -p 5000: 5000 --restart = always --name registry \
   -v `pwd` / auth: / auth \
   -e "REGISTRY_AUTH = htpasswd" \
   -e "REGISTRY_AUTH_HTPASSWD_REALM = Registry Realm" \
   -e REGISTRY_AUTH_HTPASSWD_PATH = / auth / htpasswd \
   -v `pwd` / data: / var / lib / registry \
   -v `pwd` / certs: / certs \
   -e REGISTRY_HTTP_TLS_CERTIFICATE = / certs / domain.crt \
   -e REGISTRY_HTTP_TLS_KEY = / certs / domain.key \
   registry: 2
199ad0b3591fb9613b21b1c96f017267f3c39661a7025d30df636c6805e7ab50
On 105.72, we try to push image to Registry:

$ docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
65e4158d9625: Image push failed
Head https://mydockerhub.com:5000/v2/tonybai/busybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4: no basic auth credentials
Error message prompt: Authentication failed.

Execute docker login on 72:

$ docker login mydockerhub.com:5000
Username: foo
Password:
Email: bigwhite.cn@gmail.com
WARNING: login credentials saved in /home/baiming/.docker/config.json
Login Succeeded
After successful login, push again:

$ docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
65e4158d9625: Image already exists
5506dda26018: Image already exists
latest: digest: sha256: 800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739
Push ok!

6. Management of images in Registry
As mentioned earlier, the V2 version of the Rest API can be used to query Repository and images:

$ curl --cacert domain.crt --basic --user foo: foo123 https://mydockerhub.com:5000/v2/_catalog
{"repositories": ["tonybai / busybox", "tonybai / ubuntu"]}
However, if you want to delete the Repository or the image of a tag in the Registry, v2 does not support it at present. For the reason, see Reg Instructions in istry's roadmap.

However, if your Registry's storage engine uses a local disk, there are some third-party scripts available, such as delete-docker-registry-image.

7. Summary
Less than a year after Registry2 was released, there are still many problems to be solved, such as the delete image problem. I believe these problems will be solved one by one in 2.4 and subsequent versions or a relatively ideal solution can be found.

© 2016, bigwhite. All rights reserved.