apache/beam · critical

attempted to add namespace to missing coder id: %v not in %v

Error message

attempted to add namespace to missing coder id: %v not in %v

What it means

addCoderID in the xlangx namespace package resolves a coder ID in a pipeline's Components map before rewriting its component coder IDs with an expanded namespace prefix. This panic fires when the referenced coder ID is absent from the Components.Coders map, i.e. the pipeline proto has a dangling coder reference. It is an internal invariant violation thrown as a panic, not a returnable error.

Source

Thrown at sdks/go/pkg/beam/core/runtime/xlangx/namespace.go:32

// limitations under the License.

package xlangx

import (
	"fmt"

	"github.com/apache/beam/sdks/v2/go/pkg/beam/internal/errors"
	pipepb "github.com/apache/beam/sdks/v2/go/pkg/beam/model/pipeline_v1"
)

func addCoderID(c *pipepb.Components, idMap map[string]string, cid string, newID func(string) string) string {
	if _, exists := idMap[cid]; exists {
		return idMap[cid]
	}

	coder, exists := c.Coders[cid]
	if !exists {
		panic(errors.Errorf("attempted to add namespace to missing coder id: %v not in %v", cid, c.Coders))
	}

	// Updating ComponentCoderIDs of Coder
	if coder.GetComponentCoderIds() != nil {
		var updatedComponentCoderIDs []string
		updatedComponentCoderIDs = append(updatedComponentCoderIDs, coder.ComponentCoderIds...)

		for i, ccid := range coder.ComponentCoderIds {
			updatedComponentCoderIDs[i] = addCoderID(c, idMap, ccid, newID)
		}
		coder.ComponentCoderIds = updatedComponentCoderIDs
	}

	idMap[cid] = newID(cid)

	// Updating Coders map
	c.Coders[idMap[cid]] = coder
	delete(c.Coders, cid)

View on GitHub (pinned to 12126d8942)

Solutions

1. Regenerate the expansion response: re-run the pipeline with matching, current Beam SDK versions for all involved SDKs so coder components are fully populated.
2. Inspect pipelinejson or the proto (graph.Components.Coders) and confirm every ComponentCoderId referenced by your transform exists.
3. If building pipelines manually in tests, ensure all component coders are registered in Components before namespace expansion.
4. File a bug with the Beam project if a stock Expand call triggers this with well-formed expansions.

Example fix

// before: manually built pipeline missing coder registration
pc := pipe.Components
// coders map not populated -> panic during Expand

// after: derive pipeline from a full pipeline JSON
pipe, _ := pipelinepb.FromJSON(pipelineJSON)
// ensure expand happens on a complete deserialized pipeline

for _, t := range myTransform.Subtransforms {
    for _, cid := range t.ComponentCoderIds {
        if _, ok := pipe.Components.Coders[cid]; !ok {
            return fmt.Errorf("dangling coder id %q before expansion", cid)
        }
    }
}

When it happens

Trigger: Calling xlangx.Expand on a pipeline whose graph.CrossBundles contain a ComponentCoderId pointing at a coder ID that was never added to the pipeline's Components; manually constructing or mutating pipeline protos; a bug in an expansion-response handler that drops coders.

Common situations: Cross-language transform expansion where a runner/SDK drops or rewrites components; hand-built pipelines in tests; custom external transform plumbing that constructs ExpandedTransform/pipeline pieces by hand; Beam SDK or runner version mismatches causing ID aliasing.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• attempted to add namespace to missing windowing strategy id:
• output type must be specified for sql.Transform
• res.GetError()
• invalid scope
• trigger.AfterCount(%v) must be a positive integer

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b71bfab9c9ee58f5. Report an issue: GitHub.