{"record":{"id":"a88a9bcf84c7b3cb","repo":"kubernetes/kops","slug":"error-reading-s-v-a88a9b","errorCode":null,"errorMessage":"error reading %s: %v","messagePattern":"error reading (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"util/pkg/vfs/s3fs.go","lineNumber":426,"sourceCode":"\n\tklog.V(4).Infof(\"Reading file %q\", p)\n\n\trequest := &s3.GetObjectInput{}\n\trequest.Bucket = aws.String(p.bucket)\n\trequest.Key = aws.String(p.key)\n\n\tresponse, err := client.GetObject(ctx, request)\n\tif err != nil {\n\t\tif AWSErrorCode(err) == \"NoSuchKey\" {\n\t\t\treturn 0, os.ErrNotExist\n\t\t}\n\t\treturn 0, fmt.Errorf(\"error fetching %s: %v\", p, err)\n\t}\n\tdefer response.Body.Close()\n\n\tn, err := io.Copy(out, response.Body)\n\tif err != nil {\n\t\treturn n, fmt.Errorf(\"error reading %s: %v\", p, err)\n\t}\n\treturn n, nil\n}\n\nfunc (p *S3Path) ReadDir() ([]Path, error) {\n\tctx := context.TODO()\n\tclient, err := p.client(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tprefix := p.key\n\tif prefix != \"\" && !strings.HasSuffix(prefix, \"/\") {\n\t\tprefix += \"/\"\n\t}\n\trequest := &s3.ListObjectsV2Input{}\n\trequest.Bucket = aws.String(p.bucket)\n\trequest.Prefix = aws.String(prefix)","sourceCodeStart":408,"sourceCodeEnd":444,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/util/pkg/vfs/s3fs.go#L408-L444","documentation":"S3Path.WriteToWithContext streams the S3 object body into the caller's io.Writer via io.Copy. This error wraps any failure that occurs while reading the HTTP response body after the GetObject call itself succeeded — the object exists and was fetchable, but the stream broke mid-transfer. The original AWS SDK error (network reset, timeout, checksum mismatch, connection close) is embedded in the message.","triggerScenarios":"Calling ReadFile or WriteTo on an S3Path when the GetObject response body fails partway through io.Copy — e.g. TCP connection reset, TLS handshake/keepalive timeout, proxy dropping the connection, or an SDK response-body checksum validation failure on a large object.","commonSituations":"Reading large state files (cluster specs, etcd backups) over flaky links or through corporate proxies; long-running downloads where S3 closes idle connections; container/pod network interruptions in kOps controllers mid-sync.","solutions":["Retry the read — this is almost always a transient network failure; wrap ReadFile in a bounded retry with backoff.","Inspect the wrapped error in the message for the root cause (timeout vs reset vs checksum) and fix the network path (proxy, MTU, keepalive settings).","If it happens consistently on large objects, increase client timeouts or read in ranged GetObject chunks.","Verify VPC/DNS/firewall rules allow sustained connections to the S3 regional endpoint, not just the initial request."],"exampleFix":"// before\nstateStore, err := vfs.Context.ReadFile(ctx, path)\n// after\nvar data []byte\nfor i := 0; i < 3; i++ {\n    data, err = vfs.Context.ReadFile(ctx, path)\n    if err == nil { break }\n    time.Sleep(time.Duration(1<<i) * time.Second)\n}","handlingStrategy":"retry","validationCode":"if _, err := s3Path.Path(); err == nil { /* path valid; body-read errors still possible, so preflight with a HEAD */ }\nhead, err := client.HeadObject(ctx, &s3.HeadObjectInput{Bucket: aws.String(bucket), Key: aws.String(key)})\nif err == nil && head.ContentLength != nil && *head.ContentLength > 5<<30 { // plan chunked/ranged reads for very large objects }","typeGuard":null,"tryCatchPattern":"var data []byte\nfor attempt := 0; attempt < 4; attempt++ {\n    data, err = vfs.Context.ReadFile(ctx, s3Path)\n    if err == nil || errors.Is(err, os.ErrNotExist) { break }\n    time.Sleep(time.Duration(250<<attempt) * time.Millisecond) // body reads are retryable\n}","preventionTips":["Always retry ReadFile/WriteTo — streaming body failures are transient by nature.","Keep connections alive and tune proxy/MTU settings for long S3 transfers.","Avoid reading multi-GB objects in one call; use ranged reads.","Log the wrapped cause to distinguish network resets from checksum failures."],"tags":["aws","s3","network","io","streaming"],"backgroundTag":"s3-response-body-read-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}