cocoindex-io/cocoindex · error · ValueError

VectorSchemaProvider is required for NumPy ndarray type.

Error message

VectorSchemaProvider is required for NumPy ndarray type.

What it means

_get_type_mapping maps Python types to Arrow types for a LanceDB table. A np.ndarray column requires a VectorSchemaProvider (column_spec) to supply the vector dimension; without one the mapping cannot determine the fixed-size list size, so a ValueError is raised.

Source

Thrown at python/cocoindex/connectors/lancedb/_target.py:176

    Use `LanceType` annotation with `typing.Annotated` to override the default.
    """
    type_info = analyze_type_info(python_type)

    # Check for LanceType annotation override
    for annotation in type_info.annotations:
        if isinstance(annotation, LanceType):
            return _TypeMapping(annotation.pa_type, annotation.encoder)

    base_type = type_info.base_type

    # Check direct leaf type mappings
    if base_type in _LEAF_TYPE_MAPPINGS:
        return _LEAF_TYPE_MAPPINGS[base_type]

    # NumPy ndarray: map to fixed-size list; dimension is handled at the schema layer
    if base_type is np.ndarray:
        if vector_schema is None:
            raise ValueError("VectorSchemaProvider is required for NumPy ndarray type.")

        if vector_schema.size <= 0:
            raise ValueError(f"Invalid vector dimension: {vector_schema.size}")

        # Default to float32 for vectors; use float16 for half-precision
        pa_elem = (
            pa.float16()
            if vector_schema.dtype in (np.half, np.float16)
            else pa.float32()
        )
        # Create fixed-size list type for vector
        return _TypeMapping(pa.list_(pa_elem, list_size=vector_schema.size))

    elif vector_schema is not None:
        raise ValueError(
            f"VectorSchemaProvider is only supported for NumPy ndarray type. Got type: {python_type}"
        )

View on GitHub (pinned to e84aa99b32)

Solutions

Add a column spec for the ndarray column: `column_specs={"embedding": VectorSchemaProvider(size=768)}` when constructing/from_class-ing the target.
Alternatively, store vectors as a structure the library can infer if supported by your connector version.
Check docs for VectorSchemaProvider usage in the LanceDB connector.

Example fix

// before
await LanceDbTarget.from_class(MyRecord, primary_key=["id"])  # MyRecord.embedding: np.ndarray
// after
from cocoindex.connectors.lancedb import VectorSchemaProvider
await LanceDbTarget.from_class(
    MyRecord,
    primary_key=["id"],
    column_specs={"embedding": VectorSchemaProvider(size=768)},
)

Defensive patterns

Strategy: validation

Validate before calling

from dataclasses import fields
import numpy as np
from cocoindex.connectors.lancedb import VectorSchemaProvider
ndarray_cols = [f.name for f in fields(MyRecord) if f.type is np.ndarray or f.type == np.ndarray]
missing = [c for c in ndarray_cols if c not in column_specs]
if missing:
    raise ValueError(f"Add VectorSchemaProvider column_specs for: {missing}")

Try / catch

try:
    target = await LanceDbTarget.from_class(MyRecord, primary_key=["id"], column_specs=column_specs)
except ValueError as e:
    if "VectorSchemaProvider is required" in str(e):
        ...  # add the missing column_spec and retry
    raise

Prevention

Always supply column_specs for every np.ndarray field.
Keep a single helper that builds column_specs from your embedding model's known dimension.
Read the connector docs on VectorSchemaProvider before defining record types with vectors.

When it happens

Trigger: Declaring a LanceDB target whose record type has an `np.ndarray` field, without passing a `column_specs` entry mapping that column to a VectorSchemaProvider.

Common situations: Building a LanceDB table with embedding vector columns and forgetting the column spec; examples that predate the VectorSchemaProvider requirement; relying on inference that CocoIndex intentionally does not do for ndarray dimensions.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/4cd2c811c9747c3d. Report an issue: GitHub.