apache/beam · error

failed to add scope tree

Error message

failed to add scope tree: %v

What it means

addScopeTree failed while converting an edge of a scope into proto transforms: m.addMultiEdge returned an error and it is wrapped with 'failed to add scope tree: %v' naming the scope. This occurs during graphx.Marshal when serializing a Beam pipeline graph to a ModelPipeline proto.

Solutions

1. Look at the wrapped inner error to find the actual failing edge/component (often a type or coder encoding failure)
2. Rebuild the offending transform using the standard beam.* combinators instead of hand-built graph edges
3. Verify all custom coders are registered before marshaling
4. Update the Beam SDK to a version where the used types/transforms are fully supported

Example fix

// before: hand-built edge with wrong input kind
edge.Input[0].Kind = graph.InputKind(42)
graphx.Marshal(m, graph, opts)

// after
edge.Input[0].Kind = graph.MultiMap
graphx.Marshal(m, graph, opts)

if _, err := graphx.Marshal(m, graph, opts); err != nil {
    if strings.Contains(err.Error(), "failed to add scope tree") {
        return fmt.Errorf("pipeline contains an unencodable transform/edge: %w", err)
    }
    return err
}

When it happens

Trigger: Calling graphx.Marshal (or Marshal/EncodeGraph) on a graph whose scope s contains an edge (s.Edges) that addMultiEdge cannot encode — e.g., invalid input/output kinds, unencodable types, or unregistered external coders within that edge.

Common situations: Pipelines using types that fail type-encoding (see decode/encode type errors) at submit time; custom transforms built by hand with malformed edges; SDK version issues with coders.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• bad combinefn
• bad decoding function
• bad encoding function
• bad function
• bad function type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4592a5f3d046e44f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/translate.go:287

		if ok {
			reqs = append(reqs, req)
		}
	}
	sort.Strings(reqs)
	return reqs
}

func (m *marshaller) addScopeTree(s *ScopeTree) (string, error) {
	id := scopeID(s.Scope.Scope)
	if _, exists := m.transforms[id]; exists {
		return id, nil
	}

	var subtransforms []string
	for _, edge := range s.Edges {
		ids, err := m.addMultiEdge(edge)
		if err != nil {
			return "", errors.Wrapf(err, "failed to add scope tree: %v", s)
		}
		subtransforms = append(subtransforms, ids...)
	}
	for _, tree := range s.Children {
		id, err := m.addScopeTree(tree)
		if err != nil {
			return "", errors.Wrapf(err, "failed to add scope tree: %v", s)
		}
		subtransforms = append(subtransforms, id)
	}

	metadata := m.opt.GetContextReg().ExtractTransformMetadata(s.Scope.Scope.Context)

	transform := &pipepb.PTransform{
		UniqueName:    s.Scope.Name,
		Subtransforms: subtransforms,
		EnvironmentId: m.addDefaultEnv(),
		Annotations:   metadata.Annotations,

View on GitHub (pinned to 12126d8942)