Question

Unable to verify the Docker daemon is listening

Posted December 21, 2020 2.8k views
DigitalOceanDocker

As I was following a tutorial on how to use docker-machine create I got stuck with docker-machine create –digitalocean-size “s-2vcpu-4gb” –driver digitalocean –digitalocean-access-token “token here” “server name”

Creating machine…
(keycloakexperiment1) Creating SSH key…
(keycloakexperiment1) Creating Digital Ocean droplet…
(keycloakexperiment1) Waiting for IP address to be assigned to the Droplet…
Waiting for machine to be running, this may take a few minutes…
Detecting operating system of created instance…
Waiting for SSH to be available…
Detecting the provisioner…
Provisioning with ubuntu(systemd)…
Installing Docker…
Copying certs to the local machine directory…
Copying certs to the remote machine…
Setting Docker configuration on the remote daemon…
Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded
It changes, nothing googled makes sense because any fix has to me enacted on creation.

3 comments

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
7 answers

Workaround:

docker-machine create \
        --driver digitalocean \
        --digitalocean-access-token $DOTOKEN \
        --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
        name;
  • this worked for me too yeah!

  • This workaround is not working for me. I am using Windows 10 shell. I have replaced \ with ` but still getting the following error

    Error creating machine: Error in driver during machine creation: POST https://api.digitalocean.com/v2/droplets: 422 You specified an invalid image for Droplet creation.<^>
    
    What I am doing wrong here?
    <^>
    Windows PowerShell
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    Try the new cross-platform PowerShell aka.ms/pscore6
    
    PS C:\Users\Naeem Ghumman> docker-machine create `
    >>         --driver digitalocean `
    >>         --digitalocean-access-token MY_TOKEN`
    >>         --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" `
    >>         vidly;
    Running pre-create checks...
    Creating machine...
    (vidly) Creating SSH key...
    (vidly) Creating Digital Ocean droplet...
    Error creating machine: Error in driver during machine creation: POST https://api.digitalocean.com/v2/droplets: 422 You specified an invalid image for Droplet creation.
    PS C:\Users\Naeem Ghumman>
    
    edited by bobbyiliev
    • Hi there,

      What you could do is to specify an image for your Droplets with the --digitalocean-image ubuntu-18-04-x64 flag.

      For more information, take a look at step 4 from this tutorial here:

      https://www.digitalocean.com/community/tutorials/how-to-provision-and-manage-remote-docker-hosts-with-docker-machine-on-ubuntu-18-04

      Best,
      Bobby

      by finid
      by Brian Hogan
      Docker Machine is a tool that makes it easy to provision and manage multiple Docker hosts remotely from your personal computer. In this tutorial, you'll install it on your local machine running Ubuntu 18.04 and use it to provision Dockerized DigitalOcean Droplets. If you don't have a local Ubuntu 18.04 machine, you can follow these instructions on any Ubuntu 18.04 server
      • Thank you bobyiliev for responding back.

        I tried this option but still another error

        PS C:\Users\Naeem Ghumman> docker-machine create --driver digitalocean --digitalocean-image ubuntu-18-04-x64 --digitalocean-access-token TOKEN vidly;
        Running pre-create checks...
        Creating machine...
        (vidly) Creating SSH key...
        (vidly) Creating Digital Ocean droplet...
        (vidly) Waiting for IP address to be assigned to the Droplet...
        Waiting for machine to be running, this may take a few minutes...
        Detecting operating system of created instance...
        Waiting for SSH to be available...
        Detecting the provisioner...
        Provisioning with ubuntu(systemd)...
        Installing Docker...
        Copying certs to the local machine directory...
        Copying certs to the remote machine...
        Setting Docker configuration on the remote daemon...
        Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded
        PS C:\Users\Naeem Ghumman>
        
        
        edited by bobbyiliev
        • Hello,

          This looks better, now try to also specify the --engine-install-url as mentioned in the accepted answer.

          Let me know how it goes.

          Also, make sure to remove your API token from your comment before posting the command here.

          Best,
          Bobby

          • This worked like a charm. So for windows 10 powers shell we need the below command

            docker-machine create `
                --driver digitalocean `
                --digitalocean-access-token TOKEN `
                --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" `
                --digitalocean-image ubuntu-18-04-x64 `
                vidly;
            
          • It works like that with debian image and raw docker-machine, but it does not with latest gitlab-runner.

            docker-machine create \
            --driver=digitalocean \
            --digitalocean-ssh-key-fingerprint=$DOFINGERPRINT \
            --digitalocean-tags=gitlab,runner,docker-machine \
            --digitalocean-access-token=$DOTOKEN \
            --digitalocean-image=debian-10-x64 \
            --engine-install-url=https://releases.rancher.com/install-docker/19.03.9.sh \
            testit
            

            results

            Running pre-create checks...
            Creating machine...
            (testit) Creating SSH key...
            (testit) Assuming Digital Ocean private SSH is located at ~/.ssh/id_rsa
            (testit) Creating Digital Ocean droplet...
            (testit) Waiting for IP address to be assigned to the Droplet...
            Waiting for machine to be running, this may take a few minutes...
            Detecting operating system of created instance...
            Waiting for SSH to be available...
            Error creating machine: Error detecting OS: Too many retries waiting for SSH to be available.  Last error: Maximum number of retries (60) exceeded
            

            and it fails, unless I remove fingerprint of existing key:

            docker-machine create \
            --driver=digitalocean \
            --digitalocean-tags=gitlab,runner,docker-machine \
            --digitalocean-access-token=$DOTOKEN \
            --digitalocean-image=debian-10-x64 \
            --engine-install-url=https://releases.rancher.com/install-docker/19.03.9.sh \
            testit
            

            result:

            Running pre-create checks...
            Creating machine...
            (testit) Creating SSH key...
            (testit) Creating Digital Ocean droplet...
            (testit) Waiting for IP address to be assigned to the Droplet...
            Waiting for machine to be running, this may take a few minutes...
            Detecting operating system of created instance...
            Waiting for SSH to be available...
            Detecting the provisioner...
            Provisioning with debian...
            Copying certs to the local machine directory...
            Copying certs to the remote machine...
            Setting Docker configuration on the remote daemon...
            Checking connection to Docker...
            Docker is up and running!
            To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env testit
            

            if I do same with gitlab-runner it creates a bunch of ssh keys, does not wait for ssh connection and kills instance without them finishing to start.

            gitlab-runner/config.toml

            concurrent = 4
            check_interval = 0
            
            [session_server]
              session_timeout = 1800
            
            [[runners]]
              name = "gl-runner-eu"
              url = "___"
              token = "___"
              executor = "docker+machine"
              limit = 4
              [runners.cache]
                Type = "s3"
                Shared = true
                [runners.cache.s3]
                  ServerAddress = "___"
                  AccessKey = "___"
                  SecretKey = "____"
                  BucketName = "runners"
                [runners.cache.gcs]
                [runners.cache.azure]
              [runners.docker]
                tls_verify = false
                image = "alpine:latest"
                privileged = true
                disable_entrypoint_overwrite = false
                oom_kill_disable = false
                disable_cache = false
                volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/dev/shm:/dev/shm"]
                shm_size = 0
              [runners.machine]
                IdleCount = 0
                IdleTime = 1800
                MaxBuilds = 100
                MachineDriver = "digitalocean"
                MachineName = "gitlab-runner-autoscale-%s"
                MachineOptions = [
                    "digitalocean-image=debian-10-x64",
                    "digitalocean-access-token=___",
                    "digitalocean-tags=gitlab,runner,docker-machine",
                    "digitalocean-region=lon1",
                    "digitalocean-private-networking",
                    "engine-install-url=https://releases.rancher.com/install-docker/19.03.9.sh"
                ]
            

            execution:

            service gitlab-runner start
            

            result:

            Sep 14 17:30:53 gl-runner-eu systemd[1]: gitlab-runner.service: Main process exited, code=exited, status=1/FAILURE
            Sep 14 17:30:53 gl-runner-eu systemd[1]: gitlab-runner.service: Failed with result 'exit-code'.
            Sep 14 17:31:30 gl-runner-eu systemd[1]: Stopped GitLab Runner.
            Sep 14 17:31:33 gl-runner-eu systemd[1]: Started GitLab Runner.
            Sep 14 17:31:33 gl-runner-eu gitlab-runner[11644]: Runtime platform                                    arch=amd64 os=linux pid=11644 revision=58ba2b95 version=14.2.0
            Sep 14 17:31:33 gl-runner-eu gitlab-runner[11644]: Starting multi-runner from /etc/gitlab-runner/config.toml...  builds=0
            Sep 14 17:31:33 gl-runner-eu gitlab-runner[11644]: Running in system-mode.
            Sep 14 17:31:33 gl-runner-eu gitlab-runner[11644]:
            Sep 14 17:31:33 gl-runner-eu gitlab-runner[11644]: Configuration loaded                                builds=0
            Sep 14 17:31:33 gl-runner-eu gitlab-runner[11644]: listen_address not defined, metrics & debug endpoints disabled  builds=0
            Sep 14 17:31:33 gl-runner-eu gitlab-runner[11644]: [session_server].listen_address not defined, session endpoints disabled  builds=0
            Sep 14 17:33:58 gl-runner-eu gitlab-runner[11644]: Checking for jobs... received                       job=12135 repo_url=https://gitlab.teamgate.com/dev/crm.git runner=Dc3keNaZ
            Sep 14 17:33:58 gl-runner-eu gitlab-runner[11644]: Running pre-create checks...                        driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:33:59 gl-runner-eu gitlab-runner[11644]: Checking for jobs... received                       job=12136 repo_url=https://gitlab.teamgate.com/dev/crm.git runner=Dc3keNaZ
            Sep 14 17:33:59 gl-runner-eu gitlab-runner[11644]: Running pre-create checks...                        driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:33:59 gl-runner-eu gitlab-runner[11644]: Creating machine...                                 driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:33:59 gl-runner-eu gitlab-runner[11644]: (runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07) Creating SSH key...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:33:59 gl-runner-eu gitlab-runner[11644]: Creating machine...                                 driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:33:59 gl-runner-eu gitlab-runner[11644]: (runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261) Creating SSH key...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:34:00 gl-runner-eu gitlab-runner[11644]: (runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261) Creating Digital Ocean droplet...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:34:00 gl-runner-eu gitlab-runner[11644]: (runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07) Creating Digital Ocean droplet...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:34:00 gl-runner-eu gitlab-runner[11644]: (runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07) Waiting for IP address to be assigned to the Droplet...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:34:01 gl-runner-eu gitlab-runner[11644]: (runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261) Waiting for IP address to be assigned to the Droplet...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:34:32 gl-runner-eu gitlab-runner[11644]: Waiting for machine to be running, this may take a few minutes...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:34:32 gl-runner-eu gitlab-runner[11644]: Detecting operating system of created instance...   driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:34:32 gl-runner-eu gitlab-runner[11644]: Waiting for SSH to be available...                  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:34:32 gl-runner-eu gitlab-runner[11644]: Waiting for machine to be running, this may take a few minutes...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:34:33 gl-runner-eu gitlab-runner[11644]: Detecting operating system of created instance...   driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:34:33 gl-runner-eu gitlab-runner[11644]: Waiting for SSH to be available...                  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:34:36 gl-runner-eu gitlab-runner[11644]: Detecting the provisioner...                        driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:34:36 gl-runner-eu gitlab-runner[11644]: Provisioning with debian...                         driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:34:39 gl-runner-eu gitlab-runner[11644]: Detecting the provisioner...                        driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:34:40 gl-runner-eu gitlab-runner[11644]: Provisioning with debian...                         driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640839-1e0cf261 operation=create
            Sep 14 17:36:19 gl-runner-eu gitlab-runner[11644]: Copying certs to the local machine directory...     driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:36:19 gl-runner-eu gitlab-runner[11644]: Copying certs to the remote machine...              driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:36:20 gl-runner-eu gitlab-runner[11644]: Setting Docker configuration on the remote daemon...  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:36:22 gl-runner-eu gitlab-runner[11644]: Checking connection to Docker...                    driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:36:23 gl-runner-eu gitlab-runner[11644]: Docker is up and running!                           driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:36:23 gl-runner-eu gitlab-runner[11644]: To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07  driver=digitalocean name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 operation=create
            Sep 14 17:36:23 gl-runner-eu gitlab-runner[11644]: Skipping provision retry on failed machine          error=<nil> name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07
            Sep 14 17:36:23 gl-runner-eu gitlab-runner[11644]: Machine created                                     duration=2m24.537241392s name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 now=2021-09-14 17:36:23.171337222 +0000 UTC m=+289.313873192 retries=0
            Sep 14 17:36:24 gl-runner-eu gitlab-runner[11644]: Created docker-machine                              created=2021-09-14 17:33:58.634035017 +0000 UTC m=+144.776570981 docker=tcp://159.65.80.220:2376 job=12135 name=runner-dc3kenaz-gitlab-runner-autoscale-1631640838-62270d07 now=2021-09-14 17:36:24.070535262 +0000 UTC m=+290.213071231 project=3 runner=Dc3keNaZ usedcount=1
            

            Seems like after lots of trial and error I’ve got this started…

            I’ve also tried with in marketplace docker image and failed, tried with rancheros and failed.

            So guess the smallest other image you have is debian, so I’m staying with it, as well noticed that not specifying size made it run successfully.

Turns out, the issue is with the new Docker version. You can try the older version by supplying a URL from which the Docker Engine can be downloaded like this:

docker-machine create [other opts] --engine-install-url==https://releases.rancher.com/install-docker/19.03.9.sh

Yep, the same problem today

docker-machine create --driver digitalocean --digitalocean-image ubuntu-18-04-x64 --digitalocean-size s-1vcpu-1gb --digitalocean-region ams3 --digitalocean-access-token token name

Same here today.

docker-machine create \
    --driver digitalocean \
    --digitalocean-access-token $DOTOKEN \
    --digitalocean-size "s-2vcpu-2gb" \
    eh.mdp2
Show answer This answer has been marked as resolved by naeemghumman01.