{"record":{"id":"cb5bebcbde2872b7","repo":"hashicorp/nomad","slug":"error-streaming-previous-alloc-q-for-new-alloc-q-cb5beb","errorCode":null,"errorMessage":"error streaming previous alloc %q for new alloc %q; failed reading error message: %w","messagePattern":"error streaming previous alloc %q for new alloc %q; failed reading error message: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocwatcher/alloc_watcher.go","lineNumber":607,"sourceCode":"\t\t}\n\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error streaming previous alloc %q for new alloc %q: %w\",\n\t\t\t\tp.prevAllocID, p.allocID, err)\n\t\t}\n\n\t\tif escapes, err := escapingfs.PathEscapesAllocDir(dest, \"\", hdr.Name); err != nil {\n\t\t\treturn fmt.Errorf(\"error evaluating object: %w\", err)\n\t\t} else if escapes {\n\t\t\treturn fmt.Errorf(\"archive contains object that escapes alloc dir\")\n\t\t}\n\n\t\tif hdr.Name == errorFilename {\n\t\t\t// Error snapshotting on the remote side, try to read\n\t\t\t// the message out of the file and return it.\n\t\t\terrBuf := make([]byte, int(hdr.Size))\n\t\t\tif _, err := tr.Read(errBuf); err != nil && err != io.EOF {\n\t\t\t\treturn fmt.Errorf(\"error streaming previous alloc %q for new alloc %q; failed reading error message: %w\",\n\t\t\t\t\tp.prevAllocID, p.allocID, err)\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"error streaming previous alloc %q for new alloc %q: %s\",\n\t\t\t\tp.prevAllocID, p.allocID, string(errBuf))\n\t\t}\n\n\t\t// If the header is for a directory we create the directory\n\t\tif hdr.Typeflag == tar.TypeDir {\n\t\t\tname := filepath.Join(dest, hdr.Name)\n\t\t\tos.MkdirAll(name, os.FileMode(hdr.Mode))\n\n\t\t\t// Can't change owner if not root or on Windows.\n\t\t\tif euid == 0 {\n\t\t\t\tif err := os.Chown(name, hdr.Uid, hdr.Gid); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"error chowning directory %w\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tcontinue","sourceCodeStart":589,"sourceCodeEnd":625,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocwatcher/alloc_watcher.go#L589-L625","documentation":"During migration of a previous allocation's directory, streamAllocDir receives a tar stream. If the stream contains the special error snapshot file (errorFilename), the remote side failed; Nomad tries to read the original error message out of that file. This error is thrown when reading that embedded error file itself fails (other than a clean EOF), so the real remote error message is unavailable.","triggerScenarios":"The tar stream from the remote client contains a header named errorFilename, and tr.Read into errBuf returns an error that is not io.EOF (e.g. tar stream truncated, connection reset mid-read, corrupted archive).","commonSituations":"Network interruption while migrating an allocation between nodes; the previous allocation's client crashed mid-stream; TAR reader state corruption after a prior malformed entry.","solutions":["Check connectivity between the new and previous allocation's clients and retry the migration","Verify the previous allocation's client is healthy and can finish streaming its dir snapshot","Inspect the tar stream for truncation/corruption; re-run node cleanup or GC and retry","Fall back to the returned error's wrapped cause (the %w tar read error) for the root failure"],"exampleFix":"// before\nerrBuf := make([]byte, int(hdr.Size))\nif _, err := tr.Read(errBuf); err != nil && err != io.EOF {\n    return fmt.Errorf(\"error streaming previous alloc %q for new alloc %q; failed reading error message: %w\", p.prevAllocID, p.allocID, err)\n}\n// after\nerrBuf := make([]byte, int(hdr.Size))\nif _, err := io.ReadFull(tr, errBuf); err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {\n    return fmt.Errorf(\"error streaming previous alloc %q for new alloc %q; failed reading error message: %w\", p.prevAllocID, p.allocID, err)\n}","handlingStrategy":"retry","validationCode":"// Before triggering migration, ensure the previous alloc's client is reachable\n// and the node is healthy:\n// if client.Healthy() && prevAllocNode.Ready() { proceed with migration }","typeGuard":null,"tryCatchPattern":"if err := watcher.Wait(ctx); err != nil {\n    if strings.Contains(err.Error(), \"failed reading error message\") {\n        // stream was interrupted; retry migration or start alloc without migration\n        return retryMigration(ctx, alloc)\n    }\n    return err\n}","preventionTips":["Keep stable, low-latency networking between client nodes during migrations","Avoid GC-ing or restarting the previous alloc's client mid-migration","Monitor migration failures and alert on repeated stream truncation"],"tags":["nomad","alloc-migration","tar-stream","network"],"backgroundTag":"tar-stream-read-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}