apache/beam · error

failed to create artifact role payload

Error message

failed to create artifact role payload

What it means

Thrown in newMaterializeWithClient when proto.Marshal of an ArtifactStagingToRolePayload fails while building the retrievable artifact list. The library serializes the resolved staging-to role for every dependency returned by ResolveArtifacts; a marshal failure of this simple in-memory message indicates a serialization-layer problem rather than user input. The underlying protobuf error is wrapped by this message.

Source

Thrown at sdks/go/pkg/beam/artifact/materialize.go:141

			}
			filePayload.Sha256 = typePayload.Sha256
		}
		newTypePayload, err := proto.Marshal(&filePayload)
		if err != nil {
			return nil, errors.Wrap(err, "failed to create artifact type payload")
		}
		artifacts = append(artifacts, &pipepb.ArtifactInformation{
			TypeUrn:     URNFileArtifact,
			TypePayload: newTypePayload,
			RoleUrn:     dep.RoleUrn,
			RolePayload: dep.RolePayload,
		})

		rolePayload, err := proto.Marshal(&pipepb.ArtifactStagingToRolePayload{
			StagedName: path,
		})
		if err != nil {
			return nil, errors.Wrap(err, "failed to create artifact role payload")
		}
		list = append(list, &artifact{
			client: client,
			dep: &pipepb.ArtifactInformation{
				TypeUrn:     dep.TypeUrn,
				TypePayload: dep.TypePayload,
				RoleUrn:     URNStagingTo,
				RolePayload: rolePayload,
			},
			expectedSha256: filePayload.Sha256,
		})
	}

	return artifacts, MultiRetrieve(ctx, 10, list, dest)
}

// Used for generating unique IDs. We assign uniquely generated names to staged files without staging names.
var idCounter uint64

View on GitHub (pinned to 12126d8942)

Solutions

1. Check google.golang.org/protobuf version conflicts in go.mod (go mod graph | grep protobuf) and align to a single consistent version.
2. Rebuild with a clean module cache (go clean -modcache && go mod download) to fix corrupted generated proto code.
3. Regenerate or refresh beam's pipeline_v1 protos by upgrading the beam SDK module to the latest patch version.
4. File an issue with the wrapped underlying error if it persists, as this indicates a protobuf runtime bug.

Example fix

// before: mixed protobuf modules
require google.golang.org/protobuf v1.25.0 // indirect
// after: upgrade to compatible version used by beam
require google.golang.org/protobuf v1.33.0

// Ensure a single protobuf version before running:
// go run golang.org/x/tools/cmd/goimports -l . && go mod tidy
// go list -m all | grep google.golang.org/protobuf
import "google.golang.org/protobuf/proto"
if _, err := proto.Marshal(&pipepb.ArtifactStagingToRolePayload{StagedName: "probe"}); err != nil {
	log.Fatalf("protobuf runtime broken: %v", err)
}

func canMarshalProtos() bool {
	_, err := proto.Marshal(&pipepb.ArtifactStagingToRolePayload{StagedName: "probe"})
	return err == nil
}

arts, err := artifact.Materialize(ctx, endpoint, deps, rt, dest)
if err != nil {
	if strings.Contains(err.Error(), "failed to create artifact role payload") {
		// protobuf runtime issue: fail fast with cause
		log.Fatalf("proto marshal failure (check protobuf version pinning): %v", err)
	}
	return err
}

When it happens

Trigger: Calling artifact.Materialize (or newMaterializeWithClient via tests) when proto.Marshal of &pipepb.ArtifactStagingToRolePayload{StagedName: path} fails during iteration over resolution.Replacements. Practically this only occurs if proto registry state is corrupted, a required marshaler (protoimpl) fails to initialize, or marshal options are incompatible.

Common situations: Very rare in practice: broken proto runtime registration (e.g. linking two incompatible versions of google.golang.org/protobuf), a custom binary downgraded proto library, or internal invariant violation in the protobuf runtime. Users essentially never cause this via configuration.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• empty type
• unexpected type kind %v
• unable to generate proto representation of %v
• failed to marshal payload as proto
• Could not encode message as bytes

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bb3c73f0fd760e3a. Report an issue: GitHub.