apache/beam · error

illegal multimap type

Error message

illegal multimap type: %v

What it means

makeMultiMap builds a reflect function exposing a multimap side input (func(key K) func(*V) bool style). It validates the Go type by unfolding it with funcx.UnfoldMultiMap; if the type does not match the expected multimap shape, it panics with 'illegal multimap type'. This is a wiring-time type validation, so the pipeline fails during plan construction, not during element processing.

Solutions

1. Check the DoFn parameter type: a multimap side input must be a function from key type to an iterator function, e.g. func(key K, iter func(*V) bool) or matching beam's expected multimap shape.
2. Verify the side input PCollection is a KV (PCollection<KV<K,V>>), not a plain PCollection.
3. Recreate the side input with beam.SideInput and confirm the emitted element type matches the DoFn's expectation.
4. Print the reflect type in the panic (it is included) and compare against the expected funcx.UnfoldMultiMap shape.

Example fix

// before
func (fn *f) ProcessElement(m map[string]int) { ... } // illegal multimap type
// after
func (fn *f) ProcessElement(key string, iter func(*int) bool) { ... } // valid multimap access via side input

// Validate the DoFn's multimap side-input type shape before submitting:
func isMultiMapFn(t reflect.Type) bool {
	if t.Kind() != reflect.Func || t.NumIn() != 1 || t.NumOut() != 1 {
		return false
	}
	out := t.Out(0)
	return out.Kind() == reflect.Func && out.NumIn() == 1 && out.NumOut() == 2
}

When it happens

Trigger: A DoFn parameter or graph edge typed as a multimap side input has a signature UnfoldMultiMap cannot decompose — e.g. wrong number of type parameters, a map instead of an iterable-returning function, or mismatched key/value types when makeSideInputs wires the side input.

Common situations: Mistyped DoFn signature (e.g. func(string, func(int) bool) vs expected single-arg form); passing a KV map type where an iterable factory is expected; changes to the side input declaration (beam.SideInput of PCollection<KV>) that no longer match.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• broken stream
• broken stream
• combine does not support side inputs
• expected WV coder for side input
• failed to create keyed iterable, got

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b4851564ceda1f48. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/exec/input.go:214

	return nil
}

type multiMapValue struct {
	t       reflect.Type
	keyType reflect.Type
	// These four things are needed to dynamically build the iterables
	ctx     context.Context
	adapter SideInputAdapter
	reader  StateReader
	w       typex.Window
	// fn is the actual invoked function
	fn any
}

func makeMultiMap(ctx context.Context, t reflect.Type, adapter SideInputAdapter, reader StateReader, w typex.Window) ReusableInput {
	types, ok := funcx.UnfoldMultiMap(t)
	if !ok {
		panic(fmt.Sprintf("illegal multimap type: %v", t))
	}
	mm := &multiMapValue{t: t, keyType: types[0], ctx: ctx, adapter: adapter, reader: reader, w: w}
	mm.fn = reflect.MakeFunc(t, mm.invoke).Interface()
	return mm
}

func (v *multiMapValue) Init() error {
	return nil
}

func (v *multiMapValue) Value() any {
	return v.fn
}

func (v *multiMapValue) Reset() error {
	return nil
}

View on GitHub (pinned to 12126d8942)