cocoindex-io/cocoindex · error · ValueError

Primary key column '{pk}' not found in columns: {list(self.c

Error message

Primary key column '{pk}' not found in columns: {list(self.columns.keys())}

What it means

The BigQuery target constructor validates that every column listed in primary_key exists in the declared columns dict. If a primary key name doesn't match any column, table rows could not be uniquely identified for merge/DML, so __init__ raises immediately with the full column list.

Source

Thrown at python/cocoindex/connectors/bigquery/_target.py:127

    columns: dict[str, ColumnDef]
    primary_key: list[str]
    row_type: type[RowT] | None

    def __init__(
        self,
        columns: dict[str, ColumnDef],
        primary_key: list[str],
        *,
        row_type: type[RowT] | None = None,
    ) -> None:
        self.columns = columns
        self.primary_key = primary_key
        self.row_type = row_type

        for pk in self.primary_key:
            if pk not in self.columns:
                raise ValueError(
                    f"Primary key column '{pk}' not found in columns: {list(self.columns.keys())}"
                )

    @classmethod
    async def from_class(
        cls,
        record_type: type[RowT],
        primary_key: list[str],
        *,
        column_overrides: dict[str, BigQueryType] | None = None,
    ) -> "TableSchema[RowT]":
        """
        Create a TableSchema from a record type.

        Args:
            record_type: A dataclass, NamedTuple, or Pydantic model.
            primary_key: List of column names that form the primary key.
            column_overrides: Optional per-column BigQueryType overrides.

View on GitHub (pinned to e84aa99b32)

Solutions

1. Make primary_key names match the declared columns exactly (check the list printed in the error).
2. If you used column_overrides to rename a column, update primary_key to the new name.
3. Compare against the record type fields: primary_key must be a subset of dataclass/NamedTuple/Pydantic field names.
4. Fix casing — matching is exact, so 'Id' != 'id'.

Example fix

// before
@dataclass
class Row:
    user_id: int
    name: str
target = table_target(client, "proj.ds.tbl", Row, primary_key=["id"])  # ValueError

// after
target = table_target(client, "proj.ds.tbl", Row, primary_key=["user_id"])

fields = {f.name for f in dataclasses.fields(Row)}
assert set(primary_key) <= fields, f"primary keys {set(primary_key) - fields} missing from {fields}"

try:
    target = table_target(client, table, Row, primary_key=pk)
except ValueError as e:
    logger.error("primary key mismatch: %s", e)
    raise

When it happens

Trigger: Creating the BigQuery table target (directly via __init__ or via from_class) with primary_key=['id'] while the record type/columns define fields with different names (e.g. 'user_id' or differently-cased 'ID').

Common situations: Typos in the key name; renaming a dataclass field without updating primary_key; case sensitivity mismatch between BigQuery column naming and Python field names; column overrides renaming the column but not the key.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• primary_key {primary_key!r} not found in columns ({sorted(co
• Unsupported record type: {self.record_type}
• record_type must be a record type (dataclass, NamedTuple, Py
• Invalid BigQuery {kind}: {name!r}
• Invalid BigQuery project: {project!r}

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/df11b84fefdc7f08. Report an issue: GitHub.