{"record":{"id":"f27e73c95c54a3e9","repo":"cocoindex-io/cocoindex","slug":"unexpected-column-subkey-format-sub-key-r-expe-f27e73","errorCode":null,"errorMessage":"Unexpected column subkey format: {sub_key!r}, expected to start with {_COL_SUBKEY_PREFIX!r}","messagePattern":"Unexpected column subkey format: (.+?), expected to start with (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/cocoindex/connectors/lancedb/_target.py","lineNumber":1136,"sourceCode":"        return pa.schema(fields)\n\n    async def _apply_column_actions(\n        self,\n        conn: LanceAsyncConnection,\n        table_name: str,\n        schema: TableSchema[Any],\n        column_actions: dict[str, statediff.DiffAction],\n    ) -> frozenset[str]:\n        \"\"\"Apply additive column schema changes in place.\"\"\"\n        table = await conn.open_table(table_name)\n        existing_cols = set((await table.schema()).names)\n        pk_cols = set(schema.primary_key)\n        fields_to_add: list[pa.Field] = []\n        null_backfilled_columns: set[str] = set()\n\n        for sub_key, action in column_actions.items():\n            if not sub_key.startswith(_COL_SUBKEY_PREFIX):\n                raise ValueError(\n                    f\"Unexpected column subkey format: {sub_key!r}, expected to start with {_COL_SUBKEY_PREFIX!r}\"\n                )\n\n            col_name = sub_key[len(_COL_SUBKEY_PREFIX) :]\n            if col_name in pk_cols:\n                continue\n            if col_name in existing_cols:\n                continue\n\n            desired_col = schema.columns.get(col_name)\n            if desired_col is None:\n                continue\n\n            if action in (\"insert\", \"upsert\"):\n                fields_to_add.append(\n                    # Existing rows are backfilled with null, so additive schema\n                    # evolution must materialize the new column as nullable.\n                    pa.field(col_name, desired_col.type, nullable=True)","sourceCodeStart":1118,"sourceCodeEnd":1154,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/connectors/lancedb/_target.py#L1118-L1154","documentation":"Target-state column subkeys in the LanceDB connector must be namespaced with _COL_SUBKEY_PREFIX so the engine can distinguish column-level actions from other sub-actions. During diff application, a subkey without that prefix indicates corrupted/unexpected state or a hand-built action map, so a ValueError identifies the bad subkey.","triggerScenarios":"_apply_column_actions receiving a column_actions entry whose sub_key does not start with _COL_SUBKEY_PREFIX — e.g. target-state keys constructed manually, a connector-internal format change, or stale persisted state written by an incompatible version.","commonSituations":"Upgrading CocoIndex across a subkey-format change with pre-existing persisted target state; custom tooling that writes engine state directly; internal bug injecting malformed keys.","solutions":["Clear/reset the target state for this component (drop the LanceDB table state / re-run with fresh state) so keys are regenerated in the current format.","Upgrade or align the cocoindex version between the run that wrote state and the run applying it.","If you construct actions yourself, prefix subkeys with the connector's _COL_SUBKEY_PREFIX format."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert sub_key.startswith(\"col:\"), f\"Bad column subkey: {sub_key!r}\"  # prefix per connector's _COL_SUBKEY_PREFIX","typeGuard":null,"tryCatchPattern":"try:\n    await app.update()\nexcept ValueError as e:\n    if \"Unexpected column subkey format\" in str(e):\n        ...  # reset stale target state for this component path and re-run\n    raise","preventionTips":["Don't hand-construct target-state action keys; use the connector APIs.","When upgrading cocoindex, verify stored target state compatibility; reset state if formats changed.","Run the same cocoindex version across all writers to a given target."],"tags":["python","lancedb","internal","state-format"],"backgroundTag":"invalid-argument-format","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}