hashicorp/nomad · error

failed to get docker client: %w

Error message

failed to get docker client: %w

What it means

After decoding the handle state, RecoverTask builds a Docker API client via d.getDockerClient(). If client creation fails (bad docker_endpoint, TLS setup failure, daemon init error), the error is wrapped with %w so the underlying cause is preserved and the task cannot be recovered.

Source

Thrown at drivers/docker/driver.go:255

		return nil, nil, fmt.Errorf("failed to launch docker logger process %s: %v", container.Container.ID, err)
	}

	return dlogger, pluginClient, nil
}

func (d *Driver) RecoverTask(handle *drivers.TaskHandle) error {
	if _, ok := d.tasks.Get(handle.Config.ID); ok {
		return nil
	}

	var handleState taskHandleState
	if err := handle.GetDriverState(&handleState); err != nil {
		return fmt.Errorf("failed to decode driver task state: %v", err)
	}

	dockerClient, err := d.getDockerClient()
	if err != nil {
		return fmt.Errorf("failed to get docker client: %w", err)
	}

	dockerInfo, err := dockerClient.Info(d.ctx, mclient.InfoOptions{})
	if err != nil {
		return fmt.Errorf("failed to fetch docker daemon info: %v", err)
	}

	infinityClient, err := d.getInfinityClient()
	if err != nil {
		return fmt.Errorf("failed to get docker long operations client: %w", err)
	}

	container, err := dockerClient.ContainerInspect(d.ctx, handleState.ContainerID, mclient.ContainerInspectOptions{})
	if err != nil {
		return fmt.Errorf("failed to inspect container for id %q: %v", handleState.ContainerID, err)
	}

	h := &taskHandle{

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify the Docker daemon is running (`systemctl status docker`) and the endpoint is reachable.
2. Check the client's docker_endpoint and TLS config; ensure cert/key/CA files exist and are readable by the Nomad user.
3. In containerized clients, mount /var/run/docker.sock into the Nomad container.
4. If the daemon was removed, reinstall Docker and reschedule the allocation.

Example fix

// before: client config without socket access in container
# docker run nomad-client  (no docker.sock)
// after
# docker run -v /var/run/docker.sock:/var/run/docker.sock nomad-client

cli, err := client.NewClientWithOpts(client.FromEnv)
if err != nil {
    return fmt.Errorf("docker client not constructible; check endpoint/TLS: %w", err)
}
if _, err := cli.Ping(ctx); err != nil {
    return fmt.Errorf("docker daemon unreachable: %w", err)
}

func isDockerClientErr(err error) bool { return err != nil && strings.Contains(err.Error(), "failed to get docker client") }

if err := driver.RecoverTask(handle); err != nil {
    if isDockerClientErr(err) {
        logger.Warn("docker client init failed; verify daemon/endpoint", "err", err)
        // check DOCKER_HOST/socket, then retry recovery
    } else {
        return err
    }
}

When it happens

Trigger: RecoverTask runs on a client whose Docker configuration is broken: docker_endpoint points to an unreachable socket/host, TLS cert/key/CA files are missing or unreadable, or the Docker client constructor rejects the config.

Common situations: Docker daemon stopped or uninstalled on the client after the task was started; nomad client runs in a container without /var/run/docker.sock mounted; docker_endpoint typo; TLS certs rotated/removed since the task originally started.

Related errors

• failed to reattach to docker logger process: %v
• failed to launch docker logger plugin: %v
• failed to launch docker logger process %s: %v
• failed to get docker long operations client: %w
• task with ID %q already started

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/7bd373fd11bc15a8. Report an issue: GitHub.