{"record":{"id":"0a7003065b19c468","repo":"apache/beam","slug":"failed-to-flush-chunks-for-v","errorCode":null,"errorMessage":"failed to flush chunks for %v","messagePattern":"failed to flush chunks for (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/artifact/materialize.go","lineNumber":248,"sourceCode":"\tstream, err := a.client.GetArtifact(ctx, &jobpb.GetArtifactRequest{Artifact: a.dep})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfd, err := os.OpenFile(filename, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0755)\n\tif err != nil {\n\t\treturn err\n\t}\n\tw := bufio.NewWriter(fd)\n\n\tsha256Hash, err := writeChunks(stream, w)\n\tif err != nil {\n\t\tfd.Close() // drop any buffered content\n\t\treturn errors.Wrapf(err, \"failed to retrieve chunk for %v\", filename)\n\t}\n\tif err := w.Flush(); err != nil {\n\t\tfd.Close()\n\t\treturn errors.Wrapf(err, \"failed to flush chunks for %v\", filename)\n\t}\n\tstat, _ := fd.Stat()\n\tlog.Printf(\"Downloaded: %v (sha256: %v, size: %v)\", filename, sha256Hash, stat.Size())\n\n\tif err := fd.Close(); err != nil {\n\t\treturn err\n\t}\n\n\tif isArtifactValidationEnabled(ctx) {\n\t\tif a.expectedSha256 == \"\" {\n\t\t\tlog.Printf(\"WARN: Artifact validation skipped for file: %v\", filename)\n\t\t} else if sha256Hash != a.expectedSha256 {\n\t\t\treturn errors.Errorf(\"bad SHA256 for %v: %v, want %v\", filename, sha256Hash, a.expectedSha256)\n\t\t}\n\t}\n\n\treturn nil\n}","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/artifact/materialize.go#L230-L266","documentation":"Wraps a failure of bufio.Writer.Flush in artifact.retrieve, meaning buffered artifact data could not be pushed to the file descriptor (typically a disk write error). The file is closed and the partial artifact discarded. The filename is included in the message.","triggerScenarios":"Materialize/Retrieve downloading an artifact where Flush fails: disk full, I/O error, quota exceeded, or the underlying file descriptor became invalid during the download.","commonSituations":"Worker node running out of disk space mid-download; NFS/EFS volume hiccups; container ephemeral storage quota exceeded; failing disk hardware.","solutions":["Free disk space or enlarge the volume backing dest, then rerun.","Point dest at local, reliably writable storage instead of a constrained ephemeral mount.","Check container/storage quotas for the worker (e.g. Kubernetes emptyDir sizeLimit).","Investigate storage subsystem health (dmesg/logs) if errors persist across nodes."],"exampleFix":"// before: tiny ephemeral volume as dest\nartifact.Materialize(ctx, endpoint, deps, rt, \"/tmp\")\n// after: dedicated persistent volume with adequate space\nartifact.Materialize(ctx, endpoint, deps, rt, \"/var/beam/artifacts\")","handlingStrategy":"validation","validationCode":"// Verify writable space on dest before retrieval:\nvar st syscall.Statfs_t\nif err := syscall.Statfs(dest, &st); err == nil {\n\tavail := int64(st.Bavail) * int64(st.Bsize)\n\tif avail < 512<<20 {\n\t\treturn fmt.Errorf(\"insufficient space in %s: %d bytes\", dest, avail)\n\t}\n}","typeGuard":null,"tryCatchPattern":"if err := artifact.Materialize(ctx, endpoint, deps, rt, dest); err != nil {\n\tif strings.Contains(err.Error(), \"failed to flush chunks\") {\n\t\treturn fmt.Errorf(\"storage write failure at %s: %w\", dest, err)\n\t}\n\treturn err\n}","preventionTips":["Monitor disk space on worker nodes and alert before full.","Avoid undersized Kubernetes emptyDir quotas for artifact staging.","Prefer reliable local storage over flaky network mounts for dest.","Check storage subsystem logs (dmesg) when flush errors repeat."],"tags":["go","filesystem","io","beam-artifacts"],"backgroundTag":"file-write-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}