cocoindex-io/cocoindex · error · ValueError

VectorSchemaProvider is only supported for NumPy ndarray typ

Error message

VectorSchemaProvider is only supported for NumPy ndarray type. Got type: {python_type}

What it means

VectorSchemaProvider overrides a column's Arrow type as a vector, which is only meaningful for np.ndarray columns. If a column_spec supplies a VectorSchemaProvider for any other Python type, _get_type_mapping raises a ValueError naming the offending type.

Source

Thrown at python/cocoindex/connectors/lancedb/_target.py:191

    # NumPy ndarray: map to fixed-size list; dimension is handled at the schema layer
    if base_type is np.ndarray:
        if vector_schema is None:
            raise ValueError("VectorSchemaProvider is required for NumPy ndarray type.")

        if vector_schema.size <= 0:
            raise ValueError(f"Invalid vector dimension: {vector_schema.size}")

        # Default to float32 for vectors; use float16 for half-precision
        pa_elem = (
            pa.float16()
            if vector_schema.dtype in (np.half, np.float16)
            else pa.float32()
        )
        # Create fixed-size list type for vector
        return _TypeMapping(pa.list_(pa_elem, list_size=vector_schema.size))

    elif vector_schema is not None:
        raise ValueError(
            f"VectorSchemaProvider is only supported for NumPy ndarray type. Got type: {python_type}"
        )

    # Complex types that need JSON encoding
    if isinstance(
        type_info.variant, (SequenceType, MappingType, RecordType, UnionType, AnyType)
    ):
        return _JSON_MAPPING

    # Default fallback
    return _JSON_MAPPING


class ColumnDef(NamedTuple):
    """Definition of a table column."""

    type: pa.DataType  # PyArrow type
    nullable: bool = True

View on GitHub (pinned to e84aa99b32)

Solutions

  1. Remove the VectorSchemaProvider from the non-ndarray column's spec.
  2. If the column should be a vector, change the record field type to np.ndarray and keep the provider.
  3. Match spec keys to the actual ndarray field names in your record type.

Example fix

// before
column_specs={"title": VectorSchemaProvider(size=10)}  # title is str
// after
column_specs={"embedding": VectorSchemaProvider(size=768)}  # embedding: np.ndarray
Defensive patterns

Strategy: type-guard

Validate before calling

from dataclasses import fields
import numpy as np
ndarray_names = {f.name for f in fields(MyRecord) if f.type is np.ndarray}
for k, v in column_specs.items():
    if isinstance(v, VectorSchemaProvider) and k not in ndarray_names:
        raise TypeError(f"VectorSchemaProvider on non-ndarray column: {k}")

Type guard

def is_vector_column(col: str, record_type: type) -> bool:
    import numpy as np
    from dataclasses import fields
    return any(f.name == col and f.type is np.ndarray for f in fields(record_type))

Try / catch

try:
    target = await LanceDbTarget.from_class(MyRecord, primary_key=["id"], column_specs=column_specs)
except ValueError as e:
    if "only supported for NumPy ndarray" in str(e):
        ...  # remove/relocate the misplaced spec
    raise

Prevention

When it happens

Trigger: Passing `column_specs={"some_col": VectorSchemaProvider(...)}` where `some_col` is a str/int/list/dataclass/etc. rather than np.ndarray in the record type.

Common situations: Copy-pasting a column spec from a vector column to another column; renaming fields so the vector spec lands on the wrong column; misunderstanding that the provider is only for ndarray embeddings.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/7057fe033dec0e1e. Report an issue: GitHub.