cocoindex-io/cocoindex · error

Unexpected column subkey format: {sub_key!r}, expected to st

Error message

Unexpected column subkey format: {sub_key!r}, expected to start with {_COL_SUBKEY_PREFIX!r}

What it means

Raised by _apply_column_actions when a column-level subkey in the action map does not start with the expected internal prefix (_COL_SUBKEY_PREFIX). Subkeys encode column names with a prefix so the connector can distinguish column updates from other child actions; a malformed subkey indicates an internal contract violation.

Source

Thrown at python/cocoindex/connectors/bigquery/_target.py:698

    _run_query(
        client,
        f"CREATE TABLE{if_not_exists_sql} {qualified_name} ({columns_sql})",
    )


def _apply_column_actions(
    client: Any,
    key: _TableKey,
    schema: TableSchema[Any],
    column_actions: dict[str, statediff.DiffAction],
) -> None:
    qualified_name = _qualified_table_name(key.project, key.dataset, key.table_name)
    pk_cols = set(schema.primary_key)
    non_pk_col_by_name = {n: c for n, c in schema.columns.items() if n not in pk_cols}

    for sub_key, action in column_actions.items():
        if not sub_key.startswith(_COL_SUBKEY_PREFIX):
            raise ValueError(
                f"Unexpected column subkey format: {sub_key!r}, expected to start with {_COL_SUBKEY_PREFIX!r}"
            )
        col_name = sub_key[len(_COL_SUBKEY_PREFIX) :]
        if col_name in pk_cols:
            continue

        if action == "delete":
            _run_query(
                client,
                f"ALTER TABLE {qualified_name} DROP COLUMN IF EXISTS `{col_name}`",
            )
            continue

        desired_col = non_pk_col_by_name.get(col_name)
        if desired_col is None:
            continue

        if action == "insert":

View on GitHub (pinned to e84aa99b32)

Solutions

1. Upgrade cocoindex consistently everywhere so persisted target-state subkeys match the current format.
2. Clear the stale target state (drop/re-sync the BigQuery table state) so subkeys are regenerated.
3. Do not hand-craft action maps; let the connector derive subkeys.

Example fix

// before
column_actions = {"email": action}
// after
column_actions = {"col:email": action}  # must use the internal subkey prefix

def valid_subkey(k: str) -> bool:
    return k.startswith("col:")  # matches _COL_SUBKEY_PREFIX
bad = [k for k in column_actions if not valid_subkey(k)]

def is_col_subkey(k: str) -> bool:
    return isinstance(k, str) and k.startswith("col:")

try:
    app.update()
except ValueError as e:
    if "Unexpected column subkey format" in str(e):
        # reset persisted target state and re-sync
        ...

When it happens

Trigger: A target-state key map for a BigQuery table contains a subkey that isn't `col:<name>` format, i.e. produced by a mismatched connector/handler version or hand-constructed action maps.

Common situations: Mixed cocoindex versions where target-state state persisted by an older version is replayed by a newer one; custom or monkey-patched target handlers emitting raw column names; corrupted persisted state.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Primary key column '{pk}' not found in columns: {list(self.c
• Unexpected schema type: {type(resolved_schema)}
• Unsupported record type: {self.record_type}
• record_type must be a record type (dataclass, NamedTuple, Py
• Invalid BigQuery {kind}: {name!r}

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/7651027e733ede95. Report an issue: GitHub.