{"record":{"id":"0b5a022ca1199c8e","repo":"cocoindex-io/cocoindex","slug":"unsupported-lancedb-column-action-for-in-place-evo","errorCode":null,"errorMessage":"Unsupported LanceDB column action for in-place evolution: {action!r}","messagePattern":"Unsupported LanceDB column action for in-place evolution: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/cocoindex/connectors/lancedb/_target.py","lineNumber":1160,"sourceCode":"                continue\n            if col_name in existing_cols:\n                continue\n\n            desired_col = schema.columns.get(col_name)\n            if desired_col is None:\n                continue\n\n            if action in (\"insert\", \"upsert\"):\n                fields_to_add.append(\n                    # Existing rows are backfilled with null, so additive schema\n                    # evolution must materialize the new column as nullable.\n                    pa.field(col_name, desired_col.type, nullable=True)\n                )\n                if desired_col.nullable:\n                    null_backfilled_columns.add(col_name)\n                continue\n\n            raise ValueError(\n                f\"Unsupported LanceDB column action for in-place evolution: {action!r}\"\n            )\n\n        if fields_to_add:\n            await table.add_columns(fields_to_add)\n            return frozenset(null_backfilled_columns)\n        return frozenset()\n\n    def reconcile(\n        self,\n        key: coco.StableKey,\n        desired_state: _TableSpec | coco.NonExistenceType,\n        prev_possible_records: Collection[_TableTrackingRecord],\n        prev_may_be_missing: bool,\n        /,\n    ) -> (\n        coco.TargetReconcileOutput[_TableAction, _TableTrackingRecord, _RowHandler]\n        | None","sourceCodeStart":1142,"sourceCodeEnd":1178,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/connectors/lancedb/_target.py#L1142-L1178","documentation":"During LanceDB target schema reconciliation, _apply_column_actions only supports additive in-place schema evolution ('insert'/'upsert' actions, which add new nullable columns backfilled with null). Any other DiffAction (e.g. delete, update) for a column subkey means the desired change cannot be applied in place, so the library raises rather than corrupting the table. Dropping or altering columns requires recreating the table instead.","triggerScenarios":"A diff between the declared table schema and the existing LanceDB table produces a non-additive column action (delete/update/replace) in column_actions, e.g. a column was removed or its type changed in the declared TableSpec while the table already exists.","commonSituations":"Removing or renaming a field in a row dataclass after a table already has data; changing a column's type; changing a column to/from primary key in ways the additive path skips; running an updated pipeline version against an old LanceDB table.","solutions":["Drop the existing LanceDB table (or point the target at a new table name) so it is recreated from the current declared schema","Restore the removed/changed column in the declared schema so the diff is only additive","Do a two-phase migration: add the new column (supported additive op), backfill, then manually remove the old column via LanceDB tooling","Inspect the diff actions by comparing schema.columns with the live table schema to identify the offending column"],"exampleFix":"// before\nclass Row(NamedTuple):\n    id: str\n    text: str\n    legacy_col: str  # removed later -> 'delete' action -> error\n// after\nclass Row(NamedTuple):\n    id: str\n    text: str\n# and recreate the table (or use a fresh table name) since column drops are not in-place","handlingStrategy":"validation","validationCode":"live_cols = set((await table.schema()).names)\ndeclared = set(schema.columns)\nif not (declared >= live_cols and live_cols <= declared | set()):\n    # non-additive diff possible: drops or type changes\n    if live_cols - declared:\n        raise RuntimeError(f\"columns removed; recreate table: {live_cols - declared}\")","typeGuard":"def is_additive(actions: dict[str, str]) -> bool:\n    return all(a in (\"insert\", \"upsert\") for a in actions.values())","tryCatchPattern":"try:\n    await app.update()\nexcept ValueError as e:\n    if \"Unsupported LanceDB column action\" in str(e):\n        logger.warning(\"non-additive schema change; recreating table: %s\", e)\n        await drop_lancedb_table(table_name)\n        await app.update()\n    else:\n        raise","preventionTips":["Treat declared column removals/renames as breaking changes: bump the table name or migrate explicitly","Prefer adding new columns and deprecating old ones over in-place type changes","Run schema diff checks in CI before deploying pipeline changes","Keep a version suffix in table names for destructive schema migrations"],"tags":["lancedb","schema-evolution","unsupported-operation"],"backgroundTag":"unsupported-operation","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}