Docker containers are everywhere. You can find them on Docker Hub, Quay.io,… You can easily register a working container for your purpose (eg. database, code analysis, compilation). For development, docker registry is perfect.
Now, you’ll love to use it as far as possible in your deployment chain. And probably your company, as my client, doesn’t want to have is a specific container (with codebase, data,…) exposed and public in the cloud. For sure, you can use private repositories to upload your images, but:
- It’s in the cloud
- It’s not free
My client doesn’t use Cloud at all. He always looks for On-Premise. Docker offers a complete solution for that: Docker Datacenter
So, we are one step further:
It’s in the cloud- It’s not free
We will now focus on our second point:
How can I have a docker authenticated registry On-Premise, for free?
Let’s first take a look at the Docker Registry page:
You should use the Registry if you want to:
- tightly control where your images are being stored
- fully own your images distribution pipeline
- integrate image storage and distribution tightly to your in-house development workflow
So now, how can we deploy our registry, connect to it and manage our images?
Docker Registry, Authentication: what’s behind
Again, I will take the info from Docker Registry Token Authentication:
Basically, we deploy a Token Based Authentication server “Authorization service” then configure the registry to connect to it for authentication. I will use the docker-compose v2 to run multiple containers.
The final version is on the following github: jgsqware/authenticated-registry
Token-Based Authentication server and Docker Registry configuration
Token-Based Authentication server
I will use cesanta/docker_auth/, a go implementation of authentication server.
Supported authentication methods:
- Static list of users
- Google Sign-In (incl. Google for Work / GApps for domain) (documented here)
- LDAP bind
- MongoDB user collection
The following is based on Static list of User.
First, I create the following directory structure
authenticated-registry/ ├ docker-compose.yml ├── config │ └── auth_config.yml # contain your static list of users └── ssl ├── server.key # For this purpose , └── server.pem # we will use self-signed certificate
The certificate is a self signed certificate:
$ cd authenticated-registry/ssl $ sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout server.key -out server.pem # I use auth as Common Name # auth is the URI of my authentication server
Edit the authenticated-registry/docker-compose.yml
:
version: '2' services: auth: image: cesanta/docker_auth ports: - "5001:5001" volumes: - ./config:/config:ro - ./ssl:/ssl command: /config/auth_config.yml container_name: "auth"
Edit the authenticated-registry/config/auth_config.yml
:
server: # Server settings. # Address to listen on. addr: ":5001" # TLS certificate and key. certificate: "/ssl/server.pem" key: "/ssl/server.key" token: # Settings for the tokens. issuer: "auth_service" # Must match issuer in the Registry config. expiration: 900 # Static user map. users: # Password is specified as a BCrypt hash. Use htpasswd -B to generate. "admin": password: "$2y$05$C3oDhd2O3nvcacmvGxojN.MPPvcV7LApYQU3meFMU5GeC27kb.0sK" "jgsqware": # my user password: "$2y$05$oGKwJ8QJDLBOoTBmC/EQiefIMV1N9Yt9jpX3SqMoRqZRRql6q7yam" acl: # Admin has full access to everything. - match: {account: "admin"} actions: ["*"] # Users have full right on their repository - match: {account: "/.+/", name: "${account}/*"} actions: ["*"] # Access is denied by default.
For more information about the configuration options for this authentication server, refer to the Github repo.
To generate the password with htpasswd with BCrypt hash:
$ htpasswd -nbB jgsqware jgsqware
jgsqware:$2y$05$oGKwJ8QJDLBOoTBmC/EQiefIMV1N9Yt9jpX3SqMoRqZRRql6q7yam
Registry
Docker offers his registry as a docker image. What a nice guy!
I based this setup on registry 2.1.1.
Update: Docker released the registry 2.3. It change some Manifest info and how the image is stored.
The configuration of the registry is done by environment variable.
Edit the authenticated-registry/docker-compose.yml
:
version: '2' services: auth: image: cesanta/docker_auth ports: - "5001:5001" volumes: - ./config:/config:ro - ./ssl:/ssl command: /config/auth_config.yml container_name: "auth" registry: image: registry:2.2.1 ports: - 5000:5000 volumes: - ./ssl:/ssl container_name: "registry" environment: - REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry - REGISTRY_AUTH=token - REGISTRY_AUTH_TOKEN_REALM=https://auth:5001/auth # the authentication server URI - REGISTRY_AUTH_TOKEN_SERVICE="registry" - REGISTRY_AUTH_TOKEN_ISSUER="auth_service" # Should be the same as token.issuer from authenticated-registry/config/auth_config.yml - REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE=/ssl/server.pem
Registry data persistence
I will use a Docker Volume to persist registry data.
Edit the authenticated-registry/docker-compose.yml
:
version: '2' services: auth: image: cesanta/docker_auth ports: - "5001:5001" volumes: - ./config:/config:ro - ./ssl:/ssl command: /config/auth_config.yml container_name: "auth" registry: image: registry:2.2.1 ports: - 5000:5000 volumes: - ./ssl:/ssl - registry-data:/var/lib/registry container_name: "registry" environment: - REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry - REGISTRY_AUTH=token - REGISTRY_AUTH_TOKEN_REALM=https://auth:5001/auth # the authentication server URI - REGISTRY_AUTH_TOKEN_SERVICE="registry" - REGISTRY_AUTH_TOKEN_ISSUER="auth_service" # Should be the same as token.issuer from authenticated-registry/config/auth_config.yml - REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE=/ssl/server.pem volumes: registry-data: # the real name will be <parent-folder-name>_registry-data (dash '-' int he folder name will be removed). eg. authenticatedregistry_registry-data driver: local
Let’s Run it!
To start the authenticated registry, simply run
# Run docker-compose up -d to start all container # -d is for running container as daemon $ docker-compose up -d Creating network "authenticatedregistry_default" with the default driver Creating registry Creating auth
The running container can be seen by:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6f706e4684fb cesanta/docker_auth "/auth_server /config" 47 minutes ago Up 47 minutes 0.0.0.0:5001->5001/tcp auth 073e1d94e452 registry:2.2.1 "/bin/registry /etc/d" 47 minutes ago Up 47 minutes 0.0.0.0:5000->5000/tcp registry
The created data volume cn be seen by:
$ docker volume ls DRIVER VOLUME NAME local authenticatedregistry_registry-data
Your own docker registry is now available:
- you can login,pull, push repository on localhost:5000
$ docker login localhost:5000 Username: jgsqware Password: Email: jgsqware@wemanity.com Login Succeeded
$ docker tag jgsqware/ubuntu-git localhost:5000/jgsqware/ubuntu-git $ docker push localhost:5000/jgsqware/ubuntu-git The push refers to a repository [localhost:5000/jgsqware/ubuntu-git] 3ae0de97330a: Pushing [==========================> ] 163 MB/304.5 MB 5f70bf18a086: Pushed dfd83ed44976: Pushed 217e6c75dcfc: Pushed c0b5f9221fac: Pushing [==============================================> ] 173.4 MB/187.7 MB
Backup & Restore
Backup registry data
As we have create a Docker Volume, we can backup the folder where the registry save its data to a tar file easily.
We will
- run a new container and mount the registry-data volume
- mount the local folder to
/backup
in the container. - we will compress the registry data to a tarball.
- copy the tarball to the
/backup
folder to got it on our local folder
I use the jgsqware/registry-backup image. This image to the tarball and move it to backup
the Dockerfile is simple:
FROM alpine
MAINTAINER jgsqware <jgonzalez@wemanity.com>
WORKDIR /var/lib/registry
ENTRYPOINT ["tar","cvf", "/backup/registry-data.tar","."]
$ docker run -v authenticatedregistry_registry-data:/var/lib/registry -v $(pwd):/backup jgsqware/registry-backup ./ ./docker/ ./docker/registry/ ./docker/registry/v2/ ./docker/registry/v2/repositories/ ./docker/registry/v2/repositories/jgsqware/ ./docker/registry/v2/repositories/jgsqware/ubuntu-git/ ...
You have now a file registry-data.tar
in your local folder.
Restore registry data
You can now restore the previously backuped registry data to a new registry data container.
- Run the docker-compose, you will have a registry data volume but it will be empty.
- Uncompress the
registry-data.tar
in the corresponding directory in the registry data.- Run a new container and mount the registry-data volume
- Mount the local folder to
/backup
in the container. - Uncompress the tarball to
/var/lib/registry
.
I use the jgsqware/registry-backup image. This image to the tarball and move it to backup
the Dockerfile is simple:
FROM alpine
MAINTAINER jgsqware <jgonzalez@wemanity.com>
WORKDIR /var/lib/registry
ENTRYPOINT ["tar","xvf", "/backup/registry-data.tar","."]
# The file need to be named 'registry-data.tar' $ docker run -v authenticatedregistry_registry-data:/var/lib/registry -v $(pwd):/backup jgsqware/registry-restore ./ ./docker/ ./docker/registry/ ./docker/registry/v2/ ./docker/registry/v2/repositories/ ./docker/registry/v2/repositories/jgsqware/ ./docker/registry/v2/repositories/jgsqware/ubuntu-git/ ./docker/registry/v2/repositories/jgsqware/ubuntu-git/_manifests/ ./docker/registry/v2/repositories/jgsqware/ubuntu-git/_manifests/tags/ ./docker/registry/v2/repositories/jgsqware/ubuntu-git/_manifests/tags/latest/ ./docker/registry/v2/repositories/jgsqware/ubuntu-git/_manifests/tags/latest/index/ ./docker/registry/v2/repositories/jgsqware/ubuntu-git/_manifests/tags/latest/index/sha256/ ...
Using the Docker Toolbox for OSX or Windows
Because Docker need to run on a real Linux Kernel (containerization tools exist only on Linux (for now)), you can install Docker on Windows and OSX with Docker Toolbox.
Docker Toolbox will install the client tools (docker-client, docker-compose, docker-machine) and virtualbox with boot2docker image.
boot2docker is a tiny linux os, on which docker is runnable. Docker Toolbox will configure your docker client to connect to connect to the docker-machine (here is boot2docker).
Docker Insecure Registry
To be able to use the registry in boot2docker, and because the registry is not secure by certificate (it will be in another story), we need to tell docker to allow communication with this insecure registry.
It can be done by adding --insecure-registry=<IP-OF-REGISTRY>:<PORT-OF-REGISTRY>
in the boot2docker profile.
So,
# SSH your docker-machine # docker-machine ssh <DOCKER-MACHINE-NAME> $ docker-machine ssh default # Open in vi the boot2docker profile. # profile is the configuration file used by docker when it start in boot2docker $ sudo vi /var/lib/boot2docker/profile # Add line '--insecure-registry <docker-machine-ip>:5000' in **EXTRA_ARGS** EXTRA_ARGS=' --label provider=virtualbox --insecure-registry 192.168.99.100:5000 ' # quit vi with :wq to save file # Restart docker $ sudo /etc/init.d/docker restart # Quit docker machine by CTRL-D
Open Registry port on boot2docker
You need to open the vm port5000 to put your registry on the network.
# use VBoxManage controlvm <VM-NAME> natpf1 <RULE-NAME>,tcp,<INTERFACE IP>,<VM-PORT-TO-OPEN>,,<DOCKER-PORT-TO-MAP> # Here, we are creating a rule registry in vm registry, mapping the docker port 5000 to the vm port 5000 on every interface $ VBoxManage controlvm registry natpf1 registry,tcp,0.0.0.0,5000,,5000
Now, you access over the network with:
- repository: on
<IP-OF-OSX/WINDOWS-HOST>:5000
Errors… What?
- If you try to access your insecure registry without defining it as insecure-registry, you will receive the following error when you try to push/pull$ docker pull 192.168.99.100:5000/jgsqware/ubuntu-git unable to ping registry endpoint https://192.168.99.101:5000/v0/ v2 ping attempt failed with error: Get https://192.168.99.101:5000/v2/: tls: oversized record received with length 20527 v1 ping attempt failed with error: Get https://192.168.99.101:5000/v1/_ping: tls: oversized record received with length 20527
- Sometimes, docker runs out of his mind when restarting You can see it in log, with the last line as# In your docker-machine. Remember, SSH things. $ tail -f /var/log/docker.log level=fatal msg=”Shutting down due to ServeAPI error: listen tcp 0.0.0.0:2376: bind: address already in use”If you see that, just kill docker and start it again$ sudo killall docker $ sudo /etc/init.d/docker startNow, it should be ok$ tail -f /var/log/docker.log level=info msg=”Daemon has completed initialization” level=info msg=”Docker daemon” commit=f4bf5c7 execdriver=native-0.2 graphdriver=aufs version=1.8.3
After Docker Registry: what’s next?
Now that you have your Authenticated Registry on-Premise. You can save your docker images.
The next step would be:
- Having a GUI for your registry
- Integrate the image build and deploy phase in CI
- Track vulnerabilities in your images
This post was inspired by the article: Creating Private Docker Registry 2.0 with Token Authentication Service
Other articles that might interest you: Isomorphism, and How to Bring Your Webapp to the Next Level
In a nutshell:
A Docker registry is a storage and distribution system for named Docker images;
To tightly control where your images are being stored
To fully own your images distribution pipeline
To integrate image storage and distribution tightly to your in-house development workflow
You need to deploy a Token Based Authentication server “Authorization service” then configure the registry to connect to it for authentication. You can use docker-compose v2 to run multiple containers.