cocoindex-io/cocoindex · error · ValueError

Invalid vector dimension: {vector_schema.size}

Error message

Invalid vector dimension: {vector_schema.size}

What it means

When mapping an np.ndarray column with a VectorSchemaProvider, the provider's size (vector dimension) must be positive. A zero or negative size cannot produce a valid Arrow fixed-size-list, so a ValueError names the offending dimension.

Source

Thrown at python/cocoindex/connectors/lancedb/_target.py:179

    # Check for LanceType annotation override
    for annotation in type_info.annotations:
        if isinstance(annotation, LanceType):
            return _TypeMapping(annotation.pa_type, annotation.encoder)

    base_type = type_info.base_type

    # Check direct leaf type mappings
    if base_type in _LEAF_TYPE_MAPPINGS:
        return _LEAF_TYPE_MAPPINGS[base_type]

    # NumPy ndarray: map to fixed-size list; dimension is handled at the schema layer
    if base_type is np.ndarray:
        if vector_schema is None:
            raise ValueError("VectorSchemaProvider is required for NumPy ndarray type.")

        if vector_schema.size <= 0:
            raise ValueError(f"Invalid vector dimension: {vector_schema.size}")

        # Default to float32 for vectors; use float16 for half-precision
        pa_elem = (
            pa.float16()
            if vector_schema.dtype in (np.half, np.float16)
            else pa.float32()
        )
        # Create fixed-size list type for vector
        return _TypeMapping(pa.list_(pa_elem, list_size=vector_schema.size))

    elif vector_schema is not None:
        raise ValueError(
            f"VectorSchemaProvider is only supported for NumPy ndarray type. Got type: {python_type}"
        )

    # Complex types that need JSON encoding
    if isinstance(
        type_info.variant, (SequenceType, MappingType, RecordType, UnionType, AnyType)

View on GitHub (pinned to e84aa99b32)

Solutions

  1. Pass the actual model embedding dimension, e.g. `VectorSchemaProvider(size=768)` for a 768-dim model.
  2. Validate the dimension variable before constructing the provider (`if dim <= 0: raise ...`).
  3. Trace where `size` comes from; fix the config/default that yields 0.

Example fix

// before
VectorSchemaProvider(size=len(my_embeddings))  # len == 0 before any data
// after
VectorSchemaProvider(size=768)  # static model dimension, or validate dim > 0 first
Defensive patterns

Strategy: validation

Validate before calling

dim = 768  # or from model config
assert isinstance(dim, int) and dim > 0, f"Invalid vector dim: {dim}"
spec = VectorSchemaProvider(size=dim)

Try / catch

try:
    spec = VectorSchemaProvider(size=dim)
except ValueError as e:
    if "Invalid vector dimension" in str(e):
        raise RuntimeError(f"Vector dim must be > 0, got {dim!r}; check config") from e
    raise

Prevention

When it happens

Trigger: Passing `VectorSchemaProvider(size=0)` or a negative size in column_specs for an ndarray column, typically from a variable that resolved to 0 (e.g. `len(embedding)` computed before any embedding exists, or an unset config value).

Common situations: Dimension read from an empty config/CLI value; constructing the provider from an uninitialized list/array; model config where embedding dimension defaulted to 0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/a5e77277e383e72f. Report an issue: GitHub.