{"record":{"id":"c5500b769c6ceac8","repo":"hashicorp/nomad","slug":"failed-to-launch-docker-logger-process-s-v","errorCode":null,"errorMessage":"failed to launch docker logger process %s: %v","messagePattern":"failed to launch docker logger process (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"drivers/docker/driver.go","lineNumber":237,"sourceCode":"\t\tif pluginClient != nil {\n\t\t\tpluginClient.Kill()\n\t\t}\n\t\treturn nil, nil, fmt.Errorf(\"failed to launch docker logger plugin: %v\", err)\n\t}\n\n\tif err := dlogger.Start(&docklog.StartOpts{\n\t\tEndpoint:    d.config.Endpoint,\n\t\tContainerID: container.Container.ID,\n\t\tTTY:         container.Container.Config.Tty,\n\t\tStdout:      cfg.StdoutPath,\n\t\tStderr:      cfg.StderrPath,\n\t\tTLSCert:     d.config.TLS.Cert,\n\t\tTLSKey:      d.config.TLS.Key,\n\t\tTLSCA:       d.config.TLS.CA,\n\t\tStartTime:   startTime.Unix(),\n\t}); err != nil {\n\t\tpluginClient.Kill()\n\t\treturn nil, nil, fmt.Errorf(\"failed to launch docker logger process %s: %v\", container.Container.ID, err)\n\t}\n\n\treturn dlogger, pluginClient, nil\n}\n\nfunc (d *Driver) RecoverTask(handle *drivers.TaskHandle) error {\n\tif _, ok := d.tasks.Get(handle.Config.ID); ok {\n\t\treturn nil\n\t}\n\n\tvar handleState taskHandleState\n\tif err := handle.GetDriverState(&handleState); err != nil {\n\t\treturn fmt.Errorf(\"failed to decode driver task state: %v\", err)\n\t}\n\n\tdockerClient, err := d.getDockerClient()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to get docker client: %w\", err)","sourceCodeStart":219,"sourceCodeEnd":255,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/docker/driver.go#L219-L255","documentation":"After the docker-logger plugin binary launches successfully, the driver calls dlogger.Start with the Docker endpoint, container ID, TLS material and start time. If the plugin reports an error starting log streaming for the container, the driver kills the plugin client and wraps the failure with this message, naming the container ID.","triggerScenarios":"setupNewDockerLogger (called by StartTask/RecoverTask) calls dlogger.Start and the plugin fails to attach to the container's logs: the container no longer exists or exited, the Docker endpoint/DaemonConfig is wrong, TLS certs (TLSCert/TLSKey/TLSCA) are invalid or unreadable, or the container was already torn down.","commonSituations":"Misconfigured docker_endpoint in the driver config; TLS client certs expired or not trusted by the Docker daemon; a race where the container is removed between inspect and logger start; the daemon connection was never healthy (docker.sock not mounted).","solutions":["Verify the container ID still exists (`docker inspect <id>`); if not, the task is racing with teardown and can be rescheduled.","Check the docker_endpoint config and Docker daemon reachability from the client host.","Validate TLS config (cert, key, CA) paths exist, are readable, and are signed by the daemon's CA; regenerate expired certs.","Read the plugin's underlying error in client logs to distinguish Docker API failure from TLS failure."],"exampleFix":"// before: driver config with wrong endpoint/TLS\nconfig {\n  docker_endpoint = \"tcp://127.0.0.1:2375\"\n}\n// after: correct daemon endpoint with valid TLS\nconfig {\n  docker_endpoint = \"unix:///var/run/docker.sock\"\n}","handlingStrategy":"validation","validationCode":"if cfg, err := client.ContainerInspect(ctx, containerID); err != nil || cfg.State.Running == false {\n    return fmt.Errorf(\"container %s not running; cannot start logger\", containerID)\n}","typeGuard":"func isLoggerStartErr(err error) bool { return err != nil && strings.Contains(err.Error(), \"failed to launch docker logger process\") }","tryCatchPattern":"if err := dlogger.Start(&docklog.StartOpts{...}); err != nil {\n    pluginClient.Kill()\n    if isLoggerStartErr(err) {\n        logger.Error(\"logger start failed; check daemon/TLS/container\", \"container\", containerID, \"err\", err)\n    }\n    return err\n}","preventionTips":["Validate docker_endpoint and TLS cert/key/CA paths in client config before task start.","Monitor certificate expiry for TLS-secured Docker daemons.","Confirm /var/run/docker.sock (or the TCP endpoint) is reachable from the client host.","Avoid restarting/removing containers concurrently with allocation start."],"tags":["docker","logging","tls","container","nomad-driver"],"backgroundTag":"docker-logger-start-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}