{"record":{"id":"9e0af021043cdf3f","repo":"juanfont/headscale","slug":"demultiplexing-container-logs-w","errorCode":null,"errorMessage":"demultiplexing container logs: %w","messagePattern":"demultiplexing container logs: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"cmd/hi/docker.go","lineNumber":800,"sourceCode":"\t\tFollow:     false,\n\t\tTail:       \"all\",\n\t})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"getting container logs: %w\", err)\n\t}\n\tdefer logReader.Close()\n\n\t// Create log files following the headscale naming convention\n\tstdoutPath := filepath.Join(logsDir, containerName+\".stdout.log\")\n\tstderrPath := filepath.Join(logsDir, containerName+\".stderr.log\")\n\n\t// Create buffers to capture stdout and stderr separately\n\tvar stdoutBuf, stderrBuf bytes.Buffer\n\n\t// Demultiplex the Docker logs stream to separate stdout and stderr\n\t_, err = stdcopy.StdCopy(&stdoutBuf, &stderrBuf, logReader)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"demultiplexing container logs: %w\", err)\n\t}\n\n\t// Write stdout logs\n\tif err := os.WriteFile(stdoutPath, stdoutBuf.Bytes(), 0o644); err != nil { //nolint:gosec,noinlineerr // log files should be readable\n\t\treturn fmt.Errorf(\"writing stdout log: %w\", err)\n\t}\n\n\t// Write stderr logs\n\tif err := os.WriteFile(stderrPath, stderrBuf.Bytes(), 0o644); err != nil { //nolint:gosec,noinlineerr // log files should be readable\n\t\treturn fmt.Errorf(\"writing stderr log: %w\", err)\n\t}\n\n\tif verbose {\n\t\tlog.Printf(\"Saved logs for %s: %s, %s\", containerName, stdoutPath, stderrPath)\n\t}\n\n\treturn nil\n}","sourceCodeStart":782,"sourceCodeEnd":818,"githubUrl":"https://github.com/juanfont/headscale/blob/565fd254d06c4c7f9a8cad1714a43445c79ba420/cmd/hi/docker.go#L782-L818","documentation":"stdcopy.StdCopy failed while demultiplexing the Docker log stream into stdout and stderr buffers. Docker multiplexes ContainerLogs output with an 8-byte stdcopy frame header per write; StdCopy returns an error when the stream is truncated mid-frame, contains an invalid frame, or the underlying reader fails (network drop to the daemon).","triggerScenarios":"The log stream from cli.ContainerLogs ends abruptly (daemon disconnect mid-read), a frame is malformed (partial read of the 8-byte header), or the reader returns a non-EOF error partway through StdCopy's loop.","commonSituations":"Docker daemon restarting while logs are being pulled; very large log volumes (~100 MB per run) hitting a connection reset; containers killed with SIGKILL leaving odd stream endings; inotify/filesystem issues inside the Docker VM corrupting the log file.","solutions":["Re-run the test — a truncated stream from a transient daemon drop usually does not reproduce","Check `docker logs <container>` manually to see if the stream itself is readable or truncated","Prune old runs (`control_logs/` grows ~100 MB per run) and restart Docker to clear corrupted log files","If it reproduces deterministically for one container, inspect that container's entrypoint — a container writing raw non-framed output directly to the API socket is not the issue here, but a crashing runtime can truncate mid-frame"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if _, err := stdcopy.StdCopy(&stdoutBuf, &stderrBuf, logReader); err != nil {\n    if errors.Is(err, io.ErrUnexpectedEOF) {\n        // truncated stream: keep whatever was demultiplexed and continue\n        log.Printf(\"log stream truncated for %s: %v\", containerName, err)\n    } else {\n        return fmt.Errorf(\"demultiplexing container logs: %w\", err)\n    }\n}","preventionTips":["Keep the Docker daemon stable during test runs (avoid concurrent restarts/prunes)","Save partial demultiplexed output even on StdCopy failure — partial logs still aid diagnosis","Prune control_logs/ regularly so log volume stays bounded"],"tags":["docker","stdcopy","logging","stream-demux"],"backgroundTag":null,"analyzedSha":"565fd254d06c4c7f9a8cad1714a43445c79ba420","analyzedAt":"2026-08-15T13:12:30.133Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}