apache/beam · error

zero length key

Error message

zero length key: %v %v

What it means

When Prism adds newly pending elements to a keyed stage's per-key pending map, it requires every element to have a non-empty key byte string. A zero-length key means the element's key was never extracted or was encoded with an empty key coder, which would collapse all elements onto one bogus key. The runner panics with the stage and input IDs to localize the stage.

Solutions

1. Inspect the stage (ss.ID) and its input PCollection to find the transform emitting zero-length keys
2. Fix the upstream DoFn to never emit empty string keys; filter out empty keys before the GBK stage
3. Check the key coder configuration for that PCollection; a wrong coder can decode to empty bytes
4. Validate keys in a pre-GBK ParDo and fail fast with a descriptive user error instead of a runner panic

Example fix

// before: user DoFn emits empty key
e.Emit(KV{"", value})
// after: guard against empty keys before grouping
if key == "" {
    return fmt.Errorf("cannot emit element with empty key for GBK input")
}
e.Emit(KV{key, value})

if key == "" {
    return fmt.Errorf("element key must be non-empty before GBK")
}

When it happens

Trigger: Elements arriving at a keyed (GroupByKey/CoGBK) stage where e.keyBytes is empty because the key extractor produced no bytes — e.g. an empty-key coder or a preceding stage emitting elements without keys.

Common situations: Pipelines where a DoFn emits KV with an empty/nil key in Go (e.g. KV{"", v}), custom key coders producing zero bytes, or upstream transform wiring that skips key extraction.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• couldn't decode characteristic for variant
• error decoding append bag user state window key
• error decoding residual header:
• error decoding watermarks
• error re-encoding characteristic for variant

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7fc972e1ef63a04b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go:1548

		origPending = append(origPending, e)
	}
	newPending = origPending

	if ss.pendingByKeys == nil {
		ss.pendingByKeys = map[string]*dataAndTimers{}
	}

	type windowKey struct {
		window typex.Window
		key    string
	}
	pendingWindowKeys := set[windowKey]{}

	count := 0
	for _, e := range newPending {
		count++
		if len(e.keyBytes) == 0 {
			panic(fmt.Sprintf("zero length key: %v %v", ss.ID, ss.inputID))
		}
		dnt, ok := ss.pendingByKeys[string(e.keyBytes)]
		if !ok {
			dnt = &dataAndTimers{}
			ss.pendingByKeys[string(e.keyBytes)] = dnt
		}
		heap.Push(&dnt.elements, e)

		if em.config.StreamingMode {
			// In streaming mode, we check trigger readiness on each element
			count += ss.injectTriggeredBundlesIfReady(em, e.window, string(e.keyBytes))
		} else {
			// In batch mode, we store key + window pairs here and check trigger readiness for each of them later.
			pendingWindowKeys.insert(windowKey{window: e.window, key: string(e.keyBytes)})
		}
	}
	if !em.config.StreamingMode {
		for wk := range pendingWindowKeys {

View on GitHub (pinned to 12126d8942)