cocoindex-io/cocoindex · error

PK column '{pk}' not in columns: {list(self.columns.keys())}

Error message

PK column '{pk}' not in columns: {list(self.columns.keys())}

What it means

TableSchema validates that every column listed in primary_key exists among the derived columns. When a PK name does not match any record field (after column name derivation and overrides), the schema is unusable for DDL/loads, so __init__ fails fast with ValueError.

Source

Thrown at python/cocoindex/connectors/doris/_target.py:356

@dataclass(slots=True)
class TableSchema(Generic[RowT]):
    columns: dict[str, ColumnDef]
    primary_key: list[str]
    row_type: type[RowT] | None

    def __init__(
        self,
        columns: dict[str, ColumnDef],
        primary_key: list[str],
        *,
        row_type: type[RowT] | None = None,
    ) -> None:
        self.columns = columns
        self.primary_key = primary_key
        self.row_type = row_type
        for pk in self.primary_key:
            if pk not in self.columns:
                raise ValueError(
                    f"PK column '{pk}' not in columns: {list(self.columns.keys())}"
                )

    @classmethod
    async def from_class(
        cls,
        record_type: type[RowT],
        primary_key: list[str],
        *,
        column_overrides: dict[str, DorisType | res_schema.VectorSchemaProvider]
        | None = None,
    ) -> "TableSchema[RowT]":
        if not is_record_type(record_type):
            raise TypeError(
                f"record_type must be a record type, got {type(record_type)}"
            )
        columns = await cls._columns_from_record_type(record_type, column_overrides)
        return cls(columns, primary_key, row_type=record_type)

View on GitHub (pinned to e84aa99b32)

Solutions

1. Correct the primary_key list to use exact column names present in the record type
2. If the PK field has a column name override, use the overridden column name in primary_key
3. Add the missing field to the record type or remove it from primary_key

Example fix

// before
TableSchema.from_class(Record, primary_key=["doc_id"])
class Record: id: int
// after
TableSchema.from_class(Record, primary_key=["id"])

cols = {f.name for f in dataclasses.fields(MyRecord)}
assert set(primary_key) <= cols, f"unknown PKs: {set(primary_key) - cols}"

try:
    schema = TableSchema(columns, primary_key)
except ValueError as e:
    logger.error("PK mismatch: %s", e); raise

When it happens

Trigger: Calling TableSchema(columns, primary_key=[...]) directly or from_class(..., primary_key=['id']) where 'id' is not a field name of the record type (typo, different casing, or the field was renamed/excluded by overrides).

Common situations: Renaming a dataclass field without updating primary_key; specifying the Python attribute name while the column name was overridden; passing key column names from an old schema version.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• primary_key {primary_key!r} not found in columns ({sorted(co
• Primary key column '{pk}' not found in columns: {list(self.c
• VectorSchemaProvider is required for NumPy ndarray type.
• Invalid vector dimension: {vector_schema.size}
• VectorSchemaProvider only supported for ndarray. Got: {pytho

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/530a3a94eb309e44. Report an issue: GitHub.