hashicorp/nomad · error
failed to open docker client
Error message
failed to open docker client: %v
What it means
The docker log monitoring shim (dockerLogger.Start) opens its own Docker API client via getDockerClient; failure to construct/connect is wrapped as "failed to open docker client". The logger cannot stream container logs without this client, so log collection for the task fails at startup.
Solutions
- Confirm dockerd is running and 'docker ps' works as the user Nomad runs as.
- Check the docker driver's docker_endpoint/TLS config that the logger inherits; fix wrong socket paths.
- Ensure /var/run/docker.sock is mounted and accessible when Nomad itself is containerized.
- Inspect the underlying wrapped error for connect-refused vs permission vs TLS failure and address accordingly.
Example fix
// before DOCKER_HOST="tcp://127.0.0.1:2375" (daemon listening only on socket) // after unset DOCKER_HOST # or set docker_endpoint = "unix:///var/run/docker.sock"
Defensive patterns
Strategy: validation
Validate before calling
// Ensure the endpoint the logger will inherit is reachable before starting tasks
cli, err := client.NewClientWithOpts(client.WithHost(dockerEndpoint), client.WithAPIVersionNegotiation())
if err != nil || cli.Ping(ctx) != nil {
return fmt.Errorf("docker endpoint %q unavailable for log streaming", dockerEndpoint)
} Try / catch
if err := logger.Start(opts); err != nil {
// retry with backoff — daemon may be restarting
// then fail the task's log collection explicitly
} Prevention
- Keep dockerd running under systemd with automatic restart.
- Ensure socket permissions/group membership are stable across daemon restarts.
- When Nomad is containerized, mount /var/run/docker.sock read-write for log streaming.
- Monitor daemon health so log-shim failures correlate with daemon outages.
When it happens
Trigger: Starting docker log collection when the daemon endpoint is unreachable, DOCKER_HOST/docker_endpoint is wrong, or the client can't be created from the passed DockerDriverConfig (host, TLS, auth options).
Common situations: Daemon restarted or stopped mid-task; socket permissions changed; Nomad running inside a container without the docker socket mounted; TLS certs rotated/removed from the configured paths.
Related errors
- failed to get docker client
- failed to launch docker logger process
- cannot apply allowed_modes configuration
- cannot apply allowed_modes configuration
- cannot apply allowed_modes configuration
AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04).
Data as JSON: /api/errors/2f9ab3c1098e6e7a.
Report an issue: GitHub.
Appendix: source
Thrown at drivers/docker/docklog/docker_logger.go:80
}
// dockerLogger implements the DockerLogger interface
type dockerLogger struct {
logger hclog.Logger
stdout io.WriteCloser
stderr io.WriteCloser
stdLock sync.Mutex
cancelCtx context.CancelFunc
doneCh chan interface{}
}
// Start log monitoring
func (d *dockerLogger) Start(opts *StartOpts) error {
client, err := d.getDockerClient(opts)
if err != nil {
return fmt.Errorf("failed to open docker client: %v", err)
}
ctx, cancel := context.WithCancel(context.Background())
d.cancelCtx = cancel
go func() {
defer close(d.doneCh)
stdout, stderr, err := d.openStreams(ctx, opts)
if err != nil {
d.logger.Error("log streaming ended with terminal error", "error", err)
return
}
sinceTime := time.Unix(opts.StartTime, 0)
backoff := 0.0
for {View on GitHub (pinned to 482b49bf1a)