cocoindex-io/cocoindex · error · ValueError

VectorSpecProvider is required for NumPy ndarray type.

Error message

VectorSpecProvider is required for NumPy ndarray type.

What it means

A NumPy ndarray column maps to a pgvector type, and the required vector dimension and dtype come from a VectorSpecProvider. If no vector schema is supplied for an ndarray-typed field, the type mapping cannot be built and ValueError is raised.

Source

Thrown at python/cocoindex/connectors/postgres/_target.py:287

    Use `PgType` annotation with `typing.Annotated` to override the default.
    """
    type_info = analyze_type_info(python_type)

    # Check for PgType annotation override
    for annotation in type_info.annotations:
        if isinstance(annotation, PgType):
            return _TypeMapping(annotation.pg_type, annotation.encoder)

    base_type = type_info.base_type

    # Check direct leaf type mappings
    if base_type in _LEAF_TYPE_MAPPINGS:
        return _LEAF_TYPE_MAPPINGS[base_type]

    # NumPy ndarray: map to pgvector type bases; dimension is handled at the schema layer.
    if base_type is np.ndarray:
        if vector_schema is None:
            raise ValueError("VectorSpecProvider is required for NumPy ndarray type.")
        if vector_schema.size <= 0:
            raise ValueError(f"Invalid pgvector dimension: {vector_schema.size}")

        # Default to `vector` (float32/float64/int64/etc.). Use `halfvec` for float16.
        base = "halfvec" if vector_schema.dtype in (np.half, np.float16) else "vector"
        return _TypeMapping(
            pg_type=f"{base}({vector_schema.size})", encoder=_vector_encoder
        )

    elif vector_schema is not None:
        raise ValueError(
            f"VectorSpecProvider is only supported for NumPy ndarray type. Got type: {python_type}"
        )

    # Complex types that need JSON encoding
    if isinstance(
        type_info.variant, (SequenceType, MappingType, RecordType, UnionType, AnyType)
    ):

View on GitHub (pinned to e84aa99b32)

Solutions

  1. Provide a VectorSchemaProvider for the ndarray column via column_overrides in from_class.
  2. Ensure the provider's size is a positive integer.
  3. Alternatively use a type with an explicit leaf mapping if a vector column is not intended.

Example fix

// before
target = await PgTableTarget.from_class(EmbedRow, primary_key=["id"])
// after
target = await PgTableTarget.from_class(
    EmbedRow, primary_key=["id"],
    column_overrides={"embedding": VectorSchemaProvider(size=768)})
Defensive patterns

Strategy: validation

Validate before calling

import numpy as np
for name, tp in get_type_hints(EmbedRow).items():
    if tp is np.ndarray:
        assert name in column_overrides, f"Provide VectorSchemaProvider for '{name}'"

Type guard

import numpy as np
def ndarray_columns_have_overrides(row_type: type, overrides: dict) -> bool:
    import typing
    hints = typing.get_type_hints(row_type)
    return all(
        name in overrides
        for name, tp in hints.items()
        if tp is np.ndarray
    )

Try / catch

try:
    target = await PgTableTarget.from_class(Row, primary_key=["id"], column_overrides=overrides)
except ValueError as e:
    if "VectorSpecProvider is required" in str(e):
        overrides = {**overrides, "embedding": VectorSchemaProvider(size=DIM)}

Prevention

When it happens

Trigger: Defining a record type with an np.ndarray field for TableTarget.from_class without providing column_overrides containing a VectorSchemaProvider for that column.

Common situations: Embedding columns in rows: developers add an ndarray field but forget the vector schema override, especially when the dimension is not statically known.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/12528f234e644fb2. Report an issue: GitHub.