pathwaycom/pathway · error · ValueError

Unknown source component: {self.source_component}

Error message

Unknown source component: {self.source_component}

What it means

Thrown when building a table column's engine field descriptor: schema.py maps a column's source_component (PAYLOAD_SOURCE_COMPONENT or KEY_SOURCE_COMPONENT) to api.FieldSource.PAYLOAD/KEY. Any other value reaches the else-branch and raises ValueError. In practice this means a ColumnDefinition was created with a source_component string that Pathway does not recognize.

Source

Thrown at python/pathway/internals/schema.py:734

        if self.example is not None:
            example_base64 = base64.b64encode(api.serialize(self.example)).decode(
                "UTF-8"
            )
            result["_serialized_example"] = example_base64
        return result

    @property
    def typehint(self):
        return self.dtype.typehint

    @property
    def engine_field_source(self):
        if self.source_component == PAYLOAD_SOURCE_COMPONENT:
            return api.FieldSource.PAYLOAD
        elif self.source_component == KEY_SOURCE_COMPONENT:
            return api.FieldSource.KEY
        else:
            raise ValueError(f"Unknown source component: {self.source_component}")


@dataclass(frozen=True)
class ColumnDefinition:
    primary_key: bool = False
    default_value: Any | None = _no_default_value_marker
    dtype: dt.DType | None = dt.ANY
    name: str | None = None
    append_only: bool | None = None
    description: str | None = None  # used in OpenAPI schema autogeneration
    example: Any = None  # used in OpenAPI schema autogeneration
    source_component: str = PAYLOAD_SOURCE_COMPONENT

    def __post_init__(self):
        assert self.dtype is None or isinstance(self.dtype, dt.DType)

    @classmethod
    def from_properties(cls, properties: ColumnProperties) -> ColumnDefinition:

View on GitHub (pinned to fa2f74a464)

Solutions

1. Import and use the exact constants: from pathway.internals.schema import PAYLOAD_SOURCE_COMPONENT, KEY_SOURCE_COMPONENT (or pathway.io) instead of literal strings
2. Audit any code that copies/serializes ColumnDefinition objects and verify source_component survives unchanged
3. If the schema comes from a connector with a key field, check the connector docs for how key columns must be declared rather than setting source_component manually

Example fix

// before
col = pw.ColumnDefinition(source_component="key")

// after
from pathway.internals.schema import KEY_SOURCE_COMPONENT
col = pw.ColumnDefinition(source_component=KEY_SOURCE_COMPONENT)

from pathway.internals.schema import PAYLOAD_SOURCE_COMPONENT, KEY_SOURCE_COMPONENT
_VALID = {PAYLOAD_SOURCE_COMPONENT, KEY_SOURCE_COMPONENT}
def valid_column_definition(cd) -> bool:
    return cd.source_component in _VALID

def is_known_source_component(value: str) -> bool:
    from pathway.internals.schema import PAYLOAD_SOURCE_COMPONENT, KEY_SOURCE_COMPONENT
    return value in (PAYLOAD_SOURCE_COMPONENT, KEY_SOURCE_COMPONENT)

try:
    col.engine_field_source
except ValueError as e:
    raise RuntimeError(f"Bad ColumnDefinition source_component: {e}") from e

When it happens

Trigger: Constructing a ColumnDefinition (or a schema class derived from one) with a hand-written source_component value; programmatically copying/merging ColumnDefinition objects and corrupting the field; using a connector API that lets you mark fields as coming from a key (e.g. kafka/rabbitmq style key_field) while passing an arbitrary string instead of the exported constants.

Common situations: Custom schema generation code that builds column definitions dynamically; version upgrades that renamed or moved the PAYLOAD/KEY source-component constants; typos when setting source_component='key_source' instead of importing the constant.

Related errors

• ambiguous property; schema property `{property_name}` has va
• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• Column {api.DIFF_PSEUDOCOLUMN} can only have 1 and -1 values
• schema does not match given dataframe

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/a996da9bc81d7f85. Report an issue: GitHub.