apache/beam · error

internal error

Error message

internal error

What it means

beam.createList creates a PCollection from a Go value by running a ParDo over an impulse. If the internal ParDo fails or does not return exactly one output, the library treats it as an unrecoverable internal invariant violation and panics with 'internal error' plus any underlying context.

Solutions

1. Inspect the wrapped addCreateCtx context in the panic message for the root cause and fix the element type.
2. Simplify the value passed to beam.Create (use a basic serializable type) to isolate the failure.
3. If it persists, report it as a Beam SDK bug with the failing type; this panic is not expected for valid inputs.

if err := beam.TryCreate(s, value) == nil check; err != nil { ... } // prefer TryCreate for error return

func safeCreate(s beam.Scope, v any) (col beam.PCollection, err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("create failed: %v", r)
        }
    }()
    return beam.Create(s, v), nil
}

When it happens

Trigger: Calling beam.Create/CreateList with a type that the internal TryParDo cannot process (e.g. an unsupported element type or coder failure), producing err != nil or len(ret) != 1.

Common situations: Passing types that fail schema/coder inference to beam.Create in a pipeline; a Beam SDK bug rather than user code, but usually surfaced via unusual element types.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• AfterProcessingTime trigger set without a delay or…
• array len mismatch. decoding
• At least one subtrigger required for composite triggers.
• Attempt to case match on unknown
• attempted to add namespace to missing coder id

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bdd4cf3e2ccff578. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/create.go:108

	for i, value := range values {
		if other := reflect.ValueOf(value).Type(); other != t {
			err := errors.Errorf("value %v at index %v has type %v, want %v", value, i, other, t)
			return PCollection{}, addCreateCtx(err, s)
		}
		var buf bytes.Buffer
		if err := enc.Encode(value, &buf); err != nil {
			err = errors.Wrapf(err, "marshalling of %v failed", value)
			return PCollection{}, addCreateCtx(err, s)
		}
		fn.Values = append(fn.Values, buf.Bytes())
	}

	imp := Impulse(s)

	ret, err := TryParDo(s, fn, imp, TypeDefinition{Var: TType, T: t})
	if err != nil || len(ret) != 1 {
		panic(addCreateCtx(errors.WithContext(err, "internal error"), s))
	}
	return ret[0], nil
}

// TODO(herohde) 6/26/2017: make 'create' a SDF once supported. See BEAM-2421.

type createFn struct {
	Values [][]byte    `json:"values"`
	Type   EncodedType `json:"type"`
}

func (c *createFn) ProcessElement(_ []byte, emit func(T)) error {
	dec := NewElementDecoder(c.Type.T)
	for _, val := range c.Values {
		element, err := dec.Decode(bytes.NewBuffer(val))
		if err != nil {
			return err
		}

View on GitHub (pinned to 12126d8942)