apache/beam · error

error re-encoding characteristic for variant

Error message

error re-encoding characteristic for variant %v handler %v: %v

What it means

During LoadFromYaml, each handler config YAML node is re-encoded with yaml.Marshal and then re-decoded with KnownFields(true) to validate it against the characteristic's type. If the re-encode step fails (rare, since the node came from valid YAML), prism panics.

Solutions

1. Validate the YAML file parses cleanly with a standard YAML parser before loading
2. Simplify exotic YAML constructs (anchors/aliases, tags) to plain mappings
3. Load config from a normal file rather than constructing yaml.Node values manually
4. Check go-yaml library version compatibility

Example fix

// before: exotic yaml the marshaler chokes on
handler: !custom-tag {a: 1}
// after: plain yaml
handler:
  a: 1

if _, err := yaml.Marshal(hyn); err != nil { return fmt.Errorf("handler node not marshalable: %w", err) }

defer func() { if r := recover(); r != nil { err = fmt.Errorf("config re-encode failed: %v", r) } }()

When it happens

Trigger: yaml.Marshal(hyn) returns an error while re-encoding a handler's yaml.Node during config load — typically only with invalid/marshaling-incompatible node content injected programmatically rather than parsed from a file.

Common situations: Building config nodes programmatically with unsupported content; corrupt or non-standard YAML nodes; library version differences in go-yaml behavior for exotic node types.

Related errors

• couldn't decode characteristic for variant
• error decoding append bag user state window key
• error decoding residual header:
• error decoding watermarks
• generating bundle for stage

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3e4df4efa0eb22e7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/prism/internal/config/config.go:204

	err := &unknownHandlersErr{}
	handlers := map[string]struct{}{}
	for v, hs := range r.variations {
		for hk, hyn := range hs.Handlers {
			handlers[hk] = struct{}{}

			md, ok := r.metadata[hk]
			if !ok {
				err.add(hk, v)
				continue
			}

			// Validate that handler config so we can give a good error message now.
			// We re-encode, then decode, since then we don't need to re-implement
			// the existing Known fields. Sadly, this doens't persist through
			// yaml.Node fields.
			hb, err := yaml.Marshal(hyn)
			if err != nil {
				panic(fmt.Sprintf("error re-encoding characteristic for variant %v handler %v: %v", v, hk, err))
			}
			buf := bytes.NewBuffer(hb)
			dec := yaml.NewDecoder(buf)
			dec.KnownFields(true)
			rt := md.ConfigCharacteristic()
			rtv := reflect.New(rt)
			if err := dec.Decode(rtv.Interface()); err != nil {
				return fmt.Errorf("error decoding characteristic strictly for variant %v handler %v: %v", v, hk, err)
			}

		}
	}

	if err.valid() {
		return err
	}

	r.variantIDs = maps.Keys(r.variations)

View on GitHub (pinned to 12126d8942)