apache/beam · error

chunk write failed

Error message

chunk write failed

What it means

Wrapped error from writeChunks when writing a received chunk into the buffered file writer fails during artifact streaming. The SHA256 hasher cannot fail (panics defensively), so this always reflects a real write error to the buffered writer's underlying destination later surfaced at flush, or memory/buffer issues. retrieval then wraps it with "failed to retrieve chunk for <file>".

Solutions

1. Verify free disk space and volume health where dest resides; fix underlying storage errors first.
2. Rerun the pipeline after freeing space — MultiRetrieve retries transient failures.
3. Point dest to a different, healthy local filesystem.
4. If it persists, capture the wrapped cause (printed via errors.Wrapf chain) for storage-layer diagnostics.

Example fix

// before: full disk volume
artifact.Materialize(ctx, endpoint, deps, rt, "/mnt/nearly-full")
// after: clean space or choose healthy volume
exec.Command("df", "-h", dest).Run() // verify headroom first
artifact.Materialize(ctx, endpoint, deps, rt, dest)

// Ensure destination has headroom and is writable before streaming:
probe := filepath.Join(dest, ".writeprobe")
if err := os.WriteFile(probe, make([]byte, 1<<20), 0o644); err != nil {
	return fmt.Errorf("dest cannot absorb writes: %w", err)
}
os.Remove(probe)

if err := artifact.Materialize(ctx, endpoint, deps, rt, dest); err != nil {
	if strings.Contains(err.Error(), "chunk write failed") {
		log.Printf("buffered write failure while streaming artifact; check disk/storage: %v", err)
		// surface the wrapped cause to operators
	}
	return err
}

When it happens

Trigger: During Materialize/Retrieve, bufio.Writer.Write returns an error while copying chunk.Data from the gRPC GetArtifact stream — almost always a previously failed underlying write being reported by bufio's sticky error.

Common situations: Disk full or I/O failure on the worker while streaming a large artifact; the target file descriptor became invalid (e.g. storage unmounted); extremely rare memory pressure.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• failed to flush chunks for
• error opening destination file
• failed to delete: (remove: )
• failed to retrieve chunk for
• failed to stat

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/981f33d00c75ecb7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/materialize.go:282

	return nil
}

func writeChunks(stream jobpb.ArtifactRetrievalService_GetArtifactClient, w io.Writer) (string, error) {
	sha256W := sha256.New()
	for {
		chunk, err := stream.Recv()
		if err == io.EOF {
			break
		}
		if err != nil {
			return "", err
		}
		if _, err := sha256W.Write(chunk.Data); err != nil {
			panic(err) // cannot fail
		}
		if _, err := w.Write(chunk.Data); err != nil {
			return "", errors.Wrapf(err, "chunk write failed")
		}
	}
	return hex.EncodeToString(sha256W.Sum(nil)), nil
}

func legacyMaterialize(ctx context.Context, endpoint string, rt string, dest string) ([]*pipepb.ArtifactInformation, error) {
	cc, err := grpcx.Dial(ctx, endpoint, 2*time.Minute)
	if err != nil {
		return nil, err
	}
	defer cc.Close()

	client := jobpb.NewLegacyArtifactRetrievalServiceClient(cc)

	m, err := client.GetManifest(ctx, &jobpb.GetManifestRequest{RetrievalToken: rt})
	if err != nil {
		return nil, errors.Wrap(err, "failed to get manifest")
	}

View on GitHub (pinned to 12126d8942)