Docker registry with Spaces as storage

February 12, 2018 627 views
Docker DigitalOcean Ubuntu 16.04

Hi everyone,

I'm trying to set up a private docker registry using DigitalOcean S3 compatible Spaces as a storage, but bumped into a small issue that I am not sure how to resolve. Hoping to get some thoughts on how one could resolve this, as I'm sure that I'm missing something.

I have a cluster set up on DigitalOcean, with a few managers and worker nodes. I won't go into detail on that, as it doesn't seem necessary. I am setting up my registry as a service, as I am using Swarm Mode. I'm also using Docker Flow Proxy, essentially to expose things in a neat way. So, here's how I'm deploying the service.

docker service create --name registry \
--network df-proxy --label com.df.notify=true \
--label com.df.port=5000 --label com.df.servicePath=/registry \
--label com.df.httpsOnly=true --label com.df.sslVerifyNone=true \
--secret registry_crt --secret registry_key \
--constraint "node.role==manager" --env-file registry.env registry:2

Nothing out of the ordinary here, basically exposing the service as /registry, with SSL passthrough on Docker Flow Proxy, so that it's handled by the registry, and passing out the SSL certificate along with it.

The contents of the registry.env is as follows:

REGISTRY_HTTP_ADDR=0.0.0.0:5000
REGISTRY_STORAGE=s3
REGISTRY_STORAGE_S3_ACCESSKEY=my-digitalocean-spaces-key
REGISTRY_STORAGE_S3_SECRETKEY=my-digitalocean-spaces-secret
REGISTRY_STORAGE_S3_BUCKET=bucket-name
REGISTRY_STORAGE_S3_REGION=us-west-1
REGISTRY_STORAGE_S3_REGIONENDPOINT=https://my-endpoint.digitaloceanspaces.com
REGISTRY_ENCRYPT=false
REGISTRY_HTTP_TLS_CERTIFICATE=/run/secrets/registry_crt
REGISTRY_HTTP_TLS_KEY=/run/secrets/registry_key
REGISTRY_LOG_LEVEL=info

After I set this up, I navigate into the public IP address with /registry/v2 and I get a good response, with an empty result

{ }

However, a few seconds later, the same request will start returning 503 (service unavailable) and I get a message to check /debug/health. The logs don't say much, basically I get requests that start with 200 and then change to 503.

So I went on and activated the debug health port, and got this from the output.

{"storagedriver_s3":"s3aws: NoSuchKey: \n\tstatus code: 404, request id: xxxxxxxxxxx"}

It seems the registry isn't liking something from the S3 configuration, but I can't figure out what. The access key and secret key seem to be correct, so is the bucket name and the endpoint...

Can anypoint point towards a solution or what could the problem actually be?

Cheers

2 Answers

@goncalooliveira I had a similar issue. Registry was starting and after a while it responded with 503 code (although the actual error was different). Disabling the healthcheck fixed the issue for me. You can disable it with the following env variable:

REGISTRY_HEALTH_STORAGEDRIVER_ENABLED=false

Answer is taken from here.

Hi,

Do you know where one can find more information about the following parameters:

REGISTRYSTORAGES3BUCKET=bucket-name
REGISTRY
STORAGES3REGION=us-west-1

How do they map in the context of DO Spaces?

I have a working registry (file based) running in a DO Droplet, but I'd like to store the registry in Spaces. When using the following configuration

REGISTRYSTORAGES3BUCKET: docker
REGISTRY
STORAGES3REGION: ams3

I get the following error:

registry1 | panic: s3aws: NoSuchBucket:
registry
1 | status code: 404, request id: tx00000000000000000c7b2-005ab5363c-cfe0d-ams3a

Complete config looks like:

        REGISTRY_PROXY_REMOTEURL: https://registry-1.docker.io
        REGISTRY_HEALTH_STORAGEDRIVER_ENABLED: "false"
        REGISTRY_STORAGE: s3
        REGISTRY_STORAGE_S3_ACCESSKEY: ***
        REGISTRY_STORAGE_S3_SECRETKEY: ***
        REGISTRY_STORAGE_S3_BUCKET: docker
        REGISTRY_STORAGE_S3_REGION: ams3
        REGISTRY_STORAGE_S3_REGIONENDPOINT: https://ams3.digitaloceanspaces.com

Thanks for any help,
Damien.

  • Only thing I has to change:

    REGISTRY_STORAGE_S3_BUCKET: <name of my space>
    

    Rest of the configuration is OK. So finally, quite easy.

    Best regards,
    Damien.

Have another answer? Share your knowledge.