{"record":{"id":"34a5daf6be836bf3","repo":"hashicorp/nomad","slug":"error-getting-snapshot-from-previous-alloc-q-w","errorCode":null,"errorMessage":"error getting snapshot from previous alloc %q: %w","messagePattern":"error getting snapshot from previous alloc %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocwatcher/alloc_watcher.go","lineNumber":547,"sourceCode":"\tapiConfig := nomadapi.DefaultConfig()\n\tapiConfig.Address = nodeAddr\n\tapiConfig.TLSConfig = &nomadapi.TLSConfig{\n\t\tCACert:        p.config.TLSConfig.CAFile,\n\t\tClientCert:    p.config.TLSConfig.CertFile,\n\t\tClientKey:     p.config.TLSConfig.KeyFile,\n\t\tTLSServerName: fmt.Sprintf(\"client.%s.nomad\", p.config.Region),\n\t}\n\tapiClient, err := nomadapi.NewClient(apiConfig)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\turl := fmt.Sprintf(\"/v1/client/allocation/%v/snapshot\", p.prevAllocID)\n\tqo := &nomadapi.QueryOptions{AuthToken: p.migrateToken}\n\tresp, err := apiClient.Raw().Response(url, qo)\n\tif err != nil {\n\t\tprevAllocDir.Destroy()\n\t\treturn nil, fmt.Errorf(\"error getting snapshot from previous alloc %q: %w\", p.prevAllocID, err)\n\t}\n\n\tif err := p.streamAllocDir(ctx, resp, prevAllocDir.AllocDir); err != nil {\n\t\tprevAllocDir.Destroy()\n\t\treturn nil, err\n\t}\n\n\treturn prevAllocDir, nil\n}\n\n// stream remote alloc to dir to a local path. Caller should cleanup dest on\n// error.\nfunc (p *remotePrevAlloc) streamAllocDir(ctx context.Context, resp io.ReadCloser, dest string) error {\n\tp.logger.Debug(\"streaming snapshot of previous alloc\", \"destination\", dest)\n\ttr := tar.NewReader(resp)\n\tdefer resp.Close()\n\n\t// Cache effective uid as we only run Chown if we're root","sourceCodeStart":529,"sourceCodeEnd":565,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocwatcher/alloc_watcher.go#L529-L565","documentation":"After building the local alloc dir, migrateAllocDir issues an HTTP GET to the OLD node's /v1/client/allocation/<id>/snapshot endpoint (with the migration token as AuthToken) to download a tar snapshot of the previous alloc dir. Any failure of that Raw().Response call (network error, old agent down, TLS mismatch, rejected token) is wrapped as 'error getting snapshot from previous alloc %q: %w'; the freshly built prevAllocDir is destroyed and migration fails.","triggerScenarios":"apiClient.Raw().Response(\"/v1/client/allocation/<prevAllocID>/snapshot\", qo) returns an error: previous node's HTTP API unreachable (wrong nodeAddr), agent dead or restarted, connection refused/timeout, TLS certificate/CA mismatch, or the migrateToken is invalid/expired so the request is rejected.","commonSituations":"Previous node drained or decommissioned mid-migration; firewall between nodes blocking the client HTTP port; nodeAddr resolved to a stale/wrong advertise address; TLS configured on clients but API address uses http (or vice versa); migration token expired after long queue wait.","solutions":["Verify the previous node's agent is up and its HTTP API is reachable at nodeAddr (curl the address) and that the port/firewall allow it","Check client TLS configuration consistency (ca_cert/cert/key vs http address scheme) so the HTTPS request validates","Ensure the migration token is fresh: retry the allocation/reschedule so a new migrate token is minted","Confirm the previous node is still a cluster member and nodeAddr equals its advertised HTTP address; fix advertise/rpc/serf config if stale"],"exampleFix":"// before (client config)\nhttp = \"127.0.0.1:4646\"  // unreachable from other nodes\n// after\nbind_addr = \"0.0.0.0\"\nports { http = 4646 }  # open port 4646 between client nodes for alloc migration","handlingStrategy":"retry","validationCode":"func prevNodeReachable(nodeAddr string) error {\n\tresp, err := http.Get(nodeAddr + \"/v1/agent/health\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"previous node %s unreachable: %w\", nodeAddr, err)\n\t}\n\tresp.Body.Close()\n\treturn nil\n}","typeGuard":null,"tryCatchPattern":"alloc, err := watcher.Migrate()\nif err != nil {\n\tif strings.Contains(err.Error(), \"error getting snapshot from previous alloc\") {\n\t\t// transient network/agent issue: retry with backoff before rescheduling elsewhere\n\t}\n\treturn err\n}","preventionTips":["Open the client HTTP port (default 4646) between all client nodes for alloc migration","Keep TLS config (CA, cert schemes) consistent across clients","Avoid draining or restarting a node while its allocations are still migrating","Ensure migration tokens are used promptly (short queue times) so they don't expire"],"tags":["network","http","allocation-migration","tls"],"backgroundTag":"snapshot-request-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}