apache/beam · error

unexpected SHA256 for sent chunks for

Error message

unexpected SHA256 for sent chunks for %v: %v, want %v

What it means

After streaming the file, Stage() compares the SHA256 it computed locally before upload (hash) with the hash computed while sending chunks (stagedHash). If they differ, the bytes read during upload were not identical to the file's original content, so Stage refuses to report success with 'unexpected SHA256 for sent chunks'. This is a data-integrity guard protecting against corrupted uploads or concurrent file mutation.

Solutions

1. Ensure the artifact file is final and immutable before calling StageDir/MultiStage — do not stage files still being written by a build tool.
2. Re-run staging; if the error reproduces on a stable file, verify the file content matches its expected checksum with an independent sha256sum.
3. Copy the artifact to a private temp location and stage that copy to eliminate concurrent-writer races.
4. Check computeSHA256 and the file-open order in your Beam version — if a known bug, upgrade the Beam SDK.

Example fix

// before (racy)
MultiStage(ctx, client, 10, []artifact.KeyedFile{{Key: "jar", Filename: "target/app.jar"}}, st)
// after (snapshot to immutable temp file first)
tmp, _ := os.CreateTemp("", "staged-*.jar")
src, _ := os.Open("target/app.jar")
io.Copy(tmp, src)
src.Close(); tmp.Close()
MultiStage(ctx, client, 10, []artifact.KeyedFile{{Key: "jar", Filename: tmp.Name()}}, st)

// before staging, confirm the file is stable and matches its expected checksum
info1, _ := os.Stat(filename)
time.Sleep(100 * time.Millisecond)
info2, _ := os.Stat(filename)
if !info1.ModTime().Equal(info2.ModTime()) {
	return fmt.Errorf("file %s is being modified; defer staging", filename)
}

if _, err := artifact.Stage(ctx, client, key, filename, token); err != nil {
	if strings.Contains(err.Error(), "unexpected SHA256") {
		return fmt.Errorf("artifact %s mutated during upload; rebuild and re-stage", filename)
	}
	return err
}

When it happens

Trigger: The file at filename was modified or truncated by another process between computeSHA256() and os.Open()/stageChunks; the io.Reader returned short/incorrect data; a hashing bug caused sha256W to receive different bytes than were actually hashed in computeSHA256.

Common situations: Build systems re-writing jars/binaries while Beam stages them (e.g. Gradle/Maven still writing the artifact); shared or network filesystems serving changed content; staging the same changing temp file twice concurrently.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• bad SHA256 for : , want
• chunk send failed
• failed to close stream for
• failed to scan for artifacts to stage
• failed to send chunks for

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e5ddec675abd6680. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/stage.go:169

	header := &jobpb.PutArtifactRequest{
		Content: &jobpb.PutArtifactRequest_Metadata{
			Metadata: pmd,
		},
	}
	if err := stream.Send(header); err != nil {
		stream.CloseAndRecv() // ignore error
		return nil, errors.Wrapf(err, "failed to send header for %v", filename)
	}
	stagedHash, err := stageChunks(stream, fd)
	if err != nil {
		_, errClose := stream.CloseAndRecv()
		return nil, errors.Wrapf(err, "failed to send chunks for %v; close error: %v", filename, errClose)
	}
	if resp, err := stream.CloseAndRecv(); err != nil && err != io.EOF {
		return nil, errors.Wrapf(err, "failed to close stream for %v; response: %v", filename, resp)
	}
	if hash != stagedHash {
		return nil, errors.Errorf("unexpected SHA256 for sent chunks for %v: %v, want %v", filename, stagedHash, hash)
	}
	return md, nil
}

func stageChunks(stream jobpb.LegacyArtifactStagingService_PutArtifactClient, r io.Reader) (string, error) {
	sha256W := sha256.New()
	data := make([]byte, 1<<20)
	for {
		n, err := r.Read(data)
		if n > 0 {
			if _, err := sha256W.Write(data[:n]); err != nil {
				panic(err) // cannot fail
			}

			chunk := &jobpb.PutArtifactRequest{
				Content: &jobpb.PutArtifactRequest_Data{
					Data: &jobpb.ArtifactChunk{
						Data: data[:n],

View on GitHub (pinned to 12126d8942)