What causes "forceLpCoders: coder %q not present in base map" in apache/beam?

Ensure the TestStream's coder ID is registered in the pipeline's coders map before handling the test stream

forceLpCoders: coder %q not present in base map

Error message

forceLpCoders: coder %q not present in base map

What it means

forceLpCoder (message prefix "forceLpCoders") forcibly wraps a coder in a length-prefix coder for TestStream handling in the prism runner. This error means the coder ID it was asked to force-LP does not exist in the base coder map, so no wrapping can be produced — a broken coder reference in the test stream setup.

Source

Thrown at sdks/go/pkg/beam/runners/prism/internal/coders.go:213

		bundle[lpcID] = lpc
		return lpcID, nil
	}
	return cID, nil
}

// forceLpCoder always add a new LP-coder for a given coder into the "base" map
func forceLpCoder(cID string, bundle, base map[string]*pipepb.Coder) (string, error) {
	// First check if we've already added the LP version of this coder to coders already.
	lpcID := cID + "_flp"
	// Check if we've done this one before.
	if _, ok := bundle[lpcID]; ok {
		return lpcID, nil
	}
	// Look up the canonical location.
	_, ok := base[cID]
	if !ok {
		// We messed up somewhere.
		return "", fmt.Errorf("forceLpCoders: coder %q not present in base map", cID)
	}

	lpc := &pipepb.Coder{
		Spec: &pipepb.FunctionSpec{
			Urn: urns.CoderLengthPrefix,
		},
		ComponentCoderIds: []string{cID},
	}
	bundle[lpcID] = lpc
	return lpcID, nil
}

// retrieveCoders recursively ensures that the coder along with all its direct
// and indirect component coders, are present in the `bundle` map.
// If a coder is already in `bundle`, it's skipped. Returns an error if any
// required coder ID is not found.
func retrieveCoders(cID string, bundle, base map[string]*pipepb.Coder) error {
	// Look up the canonical location.

View on GitHub (pinned to 12126d8942)

Solutions

Ensure the TestStream's coder ID is registered in the pipeline's coders map before handling the test stream
Update test fixtures to reference existing coder IDs after SDK coder changes
Compare the coder ID in the TestStream proto against the base map keys logged at failure

Defensive patterns

Strategy: validation

Validate before calling

if _, ok := base[cID]; !ok {
    return fmt.Errorf("TestStream coder %q not registered", cID)
}

Try / catch

if _, err := forceLpCoder(cID, bundle, base); err != nil {
    t.Fatalf("test stream setup failed: %v", err)
}

Prevention

Register TestStream coder IDs in pipeline components before handling
Update test fixtures when SDK coder IDs change
Assert fixture coder IDs exist in the base map in test setup

When it happens

Trigger: handleTestStream builds a TestStream source whose declared coder ID is not present in the pipeline components' coders map.

Common situations: Test stream fixtures written against coders that were renamed/removed; hand-constructed test pipelines where the TestStream coder ID was never added to the components.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/35a78cebf86769e4. Report an issue: GitHub.