cocoindex-io/cocoindex · error · ValueError

Primary key column '{pk}' not found in columns: {list(self.c

Error message

Primary key column '{pk}' not found in columns: {list(self.columns.keys())}

What it means

The LanceDB target validates that every name in primary_key refers to a column derived from the record type (plus column_specs). A primary key naming a nonexistent column makes row identity unrepresentable in the table, so __init__ raises a ValueError listing available columns.

Source

Thrown at python/cocoindex/connectors/lancedb/_target.py:252

        """
        Create a TableSchema from pre-resolved column definitions.

        For constructing from a record type, use the async classmethod
        ``from_class`` instead.

        Args:
            columns: A dict mapping column names to ColumnDef.
            primary_key: List of column names that form the primary key.
            row_type: Optional original record type.
        """
        self.columns = columns
        self.primary_key = primary_key
        self.row_type = row_type

        # Validate primary key columns exist
        for pk in self.primary_key:
            if pk not in self.columns:
                raise ValueError(
                    f"Primary key column '{pk}' not found in columns: {list(self.columns.keys())}"
                )

    @classmethod
    async def from_class(
        cls,
        record_type: type[RowT],
        primary_key: list[str],
        *,
        column_specs: dict[str, LanceType | res_schema.VectorSchemaProvider]
        | None = None,
    ) -> "TableSchema[RowT]":
        """
        Create a TableSchema from a record type (dataclass, NamedTuple, or Pydantic model).

        Python types are automatically mapped to PyArrow types.

        Args:

View on GitHub (pinned to e84aa99b32)

Solutions

1. Set primary_key to names that exactly match fields of the record type, e.g. ["id"].
2. Fix typos/case in the primary key names.
3. If the key column should exist but doesn't, add it to the record type (or via column_specs).

Example fix

// before
await LanceDbTarget.from_class(MyRecord, primary_key=["key"])  # field is named 'id'
// after
await LanceDbTarget.from_class(MyRecord, primary_key=["id"])

from dataclasses import fields
names = {f.name for f in fields(MyRecord)}
assert set(primary_key) <= names, f"Unknown pk columns: {set(primary_key) - names}; have {sorted(names)}"

try:
    target = await LanceDbTarget.from_class(MyRecord, primary_key=primary_key)
except ValueError as e:
    if "Primary key column" in str(e):
        ...  # fix primary_key names to match record fields
    raise

When it happens

Trigger: Constructing the target (directly or via from_class) with primary_key=["..."] containing a name that is not a dataclass/NamedTuple/Pydantic field of the record type and not added via column_specs, or with a typo/case mismatch.

Common situations: Renaming a record field without updating primary_key; passing the Python attribute vs serialized column name; including a column that only exists in a different record type.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Primary key column {primary_key!r} must be a scalar field, g
• zvec collections require exactly one primary key column (map
• Primary key column '{pk}' not found in columns: {list(self.c
• primary_key {primary_key!r} not found in columns ({sorted(co
• row is missing primary key field {self._primary_key!r}

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/e927f93671b95b18. Report an issue: GitHub.