apache/beam · error

external key not found %v

Error message

external key not found %v

What it means

decodeType hits a v1pb.Type_EXTERNAL type whose ExternalKey is not present in the runtime type registry (runtime.LookupType returned false). The Beam graph references an external type that must have been registered via runtime.RegisterType before decoding, and it wasn't.

Source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/serialize.go:672

		elm, err := decodeType(t.GetElement())
		if err != nil {
			wrapped := errors.Wrap(err, "bad element")
			return nil, errors.WithContextf(wrapped, "decoding type %v", t)
		}
		return reflect.PtrTo(elm), nil

	case v1pb.Type_SPECIAL:
		ret, err := decodeSpecial(t.Special)
		if err != nil {
			wrapped := errors.Wrap(err, "bad element")
			return nil, errors.WithContextf(wrapped, "decoding type %v", t)
		}
		return ret, nil

	case v1pb.Type_EXTERNAL:
		ret, ok := runtime.LookupType(t.ExternalKey)
		if !ok {
			err := errors.Errorf("external key not found %v", t.ExternalKey)
			return nil, errors.WithContextf(err, "decoding type %v", t)
		}
		return ret, nil

	default:
		err := errors.Errorf("unexpected type kind %v", t.Kind)
		return nil, errors.WithContextf(err, "failed to decode type %v", t)
	}
}

func decodeSpecial(s v1pb.Type_Special) (reflect.Type, error) {
	switch s {
	case v1pb.Type_ERROR:
		return reflectx.Error, nil
	case v1pb.Type_CONTEXT:
		return reflectx.Context, nil
	case v1pb.Type_TYPE:
		return reflectx.Type, nil

View on GitHub (pinned to 12126d8942)

Solutions

1. Add runtime.RegisterType(<externalKey>, reflect.TypeOf(...)) for the missing key, ideally in an init() of an imported package
2. Ensure the package performing the registration is imported (blank import) in the decoding binary
3. Verify the ExternalKey string in the serialized graph matches the registered key exactly
4. Align SDK versions so the same registration code that encoded the graph is present at decode time

Example fix

// before: decoding binary without registration
func main() { graphx.DecodeGraph(data) }

// after
//go:linkname nothing — just import the package that registers
import _ "myproj/beam/typeregistry" // contains init(){ runtime.RegisterType("mypkg.MyType", reflect.TypeOf(MyType{})) }

// Before decoding, verify all external keys are registered
for _, key := range collectExternalKeys(proto) {
    if _, ok := runtime.LookupType(key); !ok {
        return fmt.Errorf("external type %q not registered; add a blank import of its registering package", key)
    }
}

if err := graphx.DecodeGraph(data, &p); err != nil {
    var missing string
    if _, e := fmt.Sscanf(err.Error(), "external key not found %q", &missing); e == nil {
        return fmt.Errorf("register %q via runtime.RegisterType (blank import) before decoding", missing)
    }
    return err
}

When it happens

Trigger: Decoding a ModelPipeline containing Type_EXTERNAL whose ExternalKey was never registered in the decoding process via runtime.RegisterType(key, typ); decodeFn/decodeCustomCoder/decodeTypes propagate the failure.

Common situations: Cross-language or worker-side decoding where type registration code (usually in an init()) was not linked in; dropping an import that performed RegisterType; external type keys changed between SDK versions.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• bad input kind
• failed to decode special type, unknown type %v
• varint too long
• empty type
• error encoding byte: %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fa46e118f9d3564f. Report an issue: GitHub.