cocoindex-io/cocoindex · error

VectorSchemaProvider is required for NumPy ndarray type.

Error message

VectorSchemaProvider is required for NumPy ndarray type.

What it means

Raised by _get_type_mapping in the Doris connector when a record column is a NumPy ndarray but no VectorSchemaProvider is supplied. Vector columns need an explicit dimension/element schema to map to a Doris ARRAY<FLOAT> type, so ndarrays require a vector schema.

Source

Thrown at python/cocoindex/connectors/doris/_target.py:305

async def _get_type_mapping(
    python_type: Any, *, vector_schema: res_schema.VectorSchema | None = None
) -> _TypeMapping:
    type_info = analyze_type_info(python_type)

    for annotation in type_info.annotations:
        if isinstance(annotation, DorisType):
            return _TypeMapping(annotation.doris_type, annotation.encoder)

    base_type = type_info.base_type

    if base_type in _LEAF_TYPE_MAPPINGS:
        return _LEAF_TYPE_MAPPINGS[base_type]

    if base_type is np.ndarray:
        if vector_schema is None:
            raise ValueError("VectorSchemaProvider is required for NumPy ndarray type.")
        if vector_schema.size <= 0:
            raise ValueError(f"Invalid vector dimension: {vector_schema.size}")
        return _TypeMapping(
            "ARRAY<FLOAT>",
            lambda v: v.tolist() if hasattr(v, "tolist") else list(v),
        )
    elif vector_schema is not None:
        raise ValueError(
            f"VectorSchemaProvider only supported for ndarray. Got: {python_type}"
        )

    if isinstance(
        type_info.variant, (SequenceType, MappingType, RecordType, UnionType, AnyType)
    ):
        return _JSON_MAPPING

    return _JSON_MAPPING

View on GitHub (pinned to e84aa99b32)

Solutions

1. Provide a VectorSchemaProvider (with the correct dimension) for the ndarray column in the target declaration.
2. Convert the column to a plain list[float] if no fixed vector schema is desired.
3. Check _get_type_mapping/_columns_from_record_type calls to ensure vector_schema is threaded through for every ndarray field.

Example fix

// before
target = doris.declare_table(db, "tbl", record_type)  # record has np.ndarray field
// after
from cocoindex.resources.chunk import VectorSchemaProvider
schema = VectorSchemaProvider(size=768)
target = doris.declare_table(db, "tbl", record_type, vector_schema=schema)

from cocoindex.resources.chunk import VectorSchemaProvider
assert isinstance(vector_schema, VectorSchemaProvider), "ndarray columns need VectorSchemaProvider"

def has_vector_schema(s) -> bool:
    return s is not None and getattr(s, "size", 0) > 0

try:
    target = doris.declare_table(db, "tbl", record_type, vector_schema=vs)
except ValueError as e:
    if "VectorSchemaProvider is required" in str(e):
        # supply a VectorSchemaProvider for the ndarray column
        ...

When it happens

Trigger: Declaring a Doris target whose record type includes an np.ndarray field without passing a VectorSchemaProvider for that column (via _columns_from_record_type during target setup).

Common situations: Embedding pipelines passing raw numpy vectors to a Doris table while forgetting the vector schema; refactors that change a field from a list (auto-mapped) to ndarray; copying table declarations between connectors with different schema requirements.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Invalid vector dimension: {vector_schema.size}
• VectorSchemaProvider only supported for ndarray. Got: {pytho
• VectorSchemaProvider is required for NumPy ndarray type.
• VectorSchemaProvider is required for NumPy ndarray type.
• VectorSchemaProvider is required for NumPy ndarray type.

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/3fa82d2aef9ae8ce. Report an issue: GitHub.