apache/beam · error · CompositeTypeHintError

ShardedKey type-constraint violated. Valid object instance…

Error message

ShardedKey type-constraint violated. Valid object instance must be of type 'ShardedKey'. Instead, an instance of '%s' was received.

What it means

ShardedKeyType.type_check verifies that the runtime instance is actually a ShardedKey object before checking its inner key type. Any non-ShardedKey instance reaching a PCollection keyed with ShardedKey[K] raises CompositeTypeHintError. This guards the sharded-key encoding used by GroupByKey with sharding.

Solutions

  1. Construct keys with ShardedKey(key, shard_id) instead of tuples or plain values.
  2. Fix the PCollection type hint if the data is no longer sharded (remove ShardedKey[...] from with_output_types).
  3. Unwrap with .key/.shard_id before feeding elements into a collection expected to hold raw keys.

Example fix

// before
emitted = ShardedKeyHinted | beam.Map(lambda kv: (kv[0], kv[1]))
// after
from apache_beam.typehints.sharded_key_type import ShardedKey
emitted = ShardedKeyHinted | beam.Map(lambda sk: ShardedKey(sk.key, sk.shard_id))
Defensive patterns

Strategy: type-guard

Validate before calling

from apache_beam.typehints.sharded_key_type import ShardedKey
assert all(isinstance(k, ShardedKey) for k in sample_elements), 'elements must be ShardedKey'

Type guard

def is_sharded_key(v) -> bool:
    from apache_beam.typehints.sharded_key_type import ShardedKey
    return isinstance(v, ShardedKey)

Try / catch

try:
    run_pipeline(pipeline)
except apache_beam.typehints.TypeCheckError as e:
    log.error('element does not match ShardedKey hint: %s', e)

Prevention

When it happens

Trigger: A PCollection declared with type hint ShardedKey[...] receives, during runtime type-checking, an element that is a plain key, tuple, or other object not constructed via ShardedKey().

Common situations: Users build sharded keys manually as tuples (key, shard) instead of ShardedKey(key, shard), or a downstream map emits the unwrapped key back into a collection still hinted as ShardedKey.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bc24c6209b5048e6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/typehints/sharded_key_type.py:53

class ShardedKeyTypeConstraint(typehints.TypeConstraint,
                               metaclass=typehints.GetitemConstructor):
  def __init__(self, key_type):
    typehints.validate_composite_type_param(
        key_type, error_msg_prefix='Parameter to ShardedKeyType hint')
    self.key_type = typehints.normalize(key_type)

  def _inner_types(self):
    yield self.key_type

  def _consistent_with_check_(self, sub):
    return (
        isinstance(sub, self.__class__) and
        typehints.is_consistent_with(sub.key_type, self.key_type))

  def type_check(self, instance):
    if not isinstance(instance, ShardedKey):
      raise typehints.CompositeTypeHintError(
          "ShardedKey type-constraint violated. Valid object instance "
          "must be of type 'ShardedKey'. Instead, an instance of '%s' "
          "was received." % (instance.__class__.__name__))

    try:
      typehints.check_constraint(self.key_type, instance.key)
    except (typehints.CompositeTypeHintError, typehints.SimpleTypeHintError):
      raise typehints.CompositeTypeHintError(
          "%s type-constraint violated. The type of key in 'ShardedKey' "
          "is incorrect. Expected an instance of type '%s', "
          "instead received an instance of type '%s'." %
          (repr(self), repr(self.key_type), instance.key.__class__.__name__))

  def match_type_variables(self, concrete_type):
    if isinstance(concrete_type, ShardedKeyTypeConstraint):
      return typehints.match_type_variables(
          self.key_type, concrete_type.key_type)
    return {}

View on GitHub (pinned to 12126d8942)