apache/beam · error

artifact not staged

Error message

artifact %v not staged

What it means

matchLocations verifies that every artifact listed in the job manifest has a corresponding blob already staged in GCS. If an artifact name is absent from the staging server's blob map, it cannot produce a ProxyManifest location, so it fails fast with this error. CommitManifest calls it when finalizing the staged manifest.

Solutions

1. Stage all manifest artifacts via PutArtifact on the same StagingServer instance before calling CommitManifest
2. Verify artifact names in the manifest exactly match md.Name values sent during PutArtifact
3. If the server restarts, re-stage the artifacts before committing the manifest

Example fix

// before: committing manifest without staging
proxy.CommitManifest(ctx, artifacts) // artifact 'foo.jar' never staged

// after: stage first, then commit
for _, a := range toStage { stream.PutArtifact(ctx, a) }
proxy.CommitManifest(ctx, artifacts)

// before CommitManifest
for _, a := range artifacts {
    if _, ok := serverBlobs[a.Name]; !ok {
        return fmt.Errorf("artifact %q missing; stage it via PutArtifact first", a.Name)
    }
}

func isStaged(name string, blobs map[string]staged) bool { _, ok := blobs[name]; return ok }

if err := CommitManifest(ctx, req); err != nil {
    if strings.Contains(err.Error(), "not staged") {
        // re-stage the artifact, then retry commit
    }
    return err
}

When it happens

Trigger: Calling CommitManifest when the manifest references an artifact whose name was never staged via PutArtifact on this StagingServer, or after the staging server was restarted losing in-memory blob state.

Common situations: Job submitted referencing artifacts staged by a different worker; StagingServer restarted between PutArtifact and CommitManifest; artifact names in the manifest don't match names used at staging time (typos, path prefix differences).

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

• empty chunk
• failed to receive header
• failed to stage artifact
• invalid SHA256 for artifact
• staged artifact for has invalid SHA256: , want

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fae140a6bd7c4b59. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/gcsproxy/staging.go:106

		return nil, errors.Wrap(err, "failed to write manifest")
	}

	// Commit returns the location of the manifest as the token, which can
	// then be used to configure the retrieval proxy. It is redundant right
	// now, but would be needed for a staging server that serves multiple
	// jobs. Such a server would also use the ID sent with each request.

	return &jobpb.CommitManifestResponse{RetrievalToken: gcsx.MakeObject(s.bucket, s.manifest)}, nil
}

// matchLocations ensures that all artifacts have been staged and have valid
// content. It is fine for staged artifacts to not appear in the manifest.
func matchLocations(artifacts []*jobpb.ArtifactMetadata, blobs map[string]staged) ([]*jobpb.ProxyManifest_Location, error) {
	var loc []*jobpb.ProxyManifest_Location
	for _, a := range artifacts {
		info, ok := blobs[a.Name]
		if !ok {
			return nil, errors.Errorf("artifact %v not staged", a.Name)
		}
		if a.Sha256 == "" {
			a.Sha256 = info.hash
		}
		if info.hash != a.Sha256 {
			return nil, errors.Errorf("staged artifact for %v has invalid SHA256: %v, want %v", a.Name, info.hash, a.Sha256)
		}

		loc = append(loc, &jobpb.ProxyManifest_Location{Name: a.Name, Uri: info.object})
	}
	return loc, nil
}

// PutArtifact stores the given artifact in GCS.
func (s *StagingServer) PutArtifact(ps jobpb.LegacyArtifactStagingService_PutArtifactServer) error {
	// Read header

	header, err := ps.Recv()

View on GitHub (pinned to 12126d8942)