cocoindex-io/cocoindex · error

Invalid vector dimension: {vector_schema.size}

Error message

Invalid vector dimension: {vector_schema.size}

What it means

Validation during Doris column type resolution in _get_type_mapping. An np.ndarray field maps to a Doris vector/array column whose element count must come from a VectorSchema (via an Annotated NDArray or column override). When a vector annotation exists but declares a non-positive/invalid size, no valid DORIS type can be emitted, so this ValueError fires. Declare the vector field with a VectorSchema specifying a positive dimension.

Source

Thrown at python/cocoindex/connectors/doris/_target.py:307

async def _get_type_mapping(
    python_type: Any, *, vector_schema: res_schema.VectorSchema | None = None
) -> _TypeMapping:
    type_info = analyze_type_info(python_type)

    for annotation in type_info.annotations:
        if isinstance(annotation, DorisType):
            return _TypeMapping(annotation.doris_type, annotation.encoder)

    base_type = type_info.base_type

    if base_type in _LEAF_TYPE_MAPPINGS:
        return _LEAF_TYPE_MAPPINGS[base_type]

    if base_type is np.ndarray:
        if vector_schema is None:
            raise ValueError("VectorSchemaProvider is required for NumPy ndarray type.")
        if vector_schema.size <= 0:
            raise ValueError(f"Invalid vector dimension: {vector_schema.size}")
        return _TypeMapping(
            "ARRAY<FLOAT>",
            lambda v: v.tolist() if hasattr(v, "tolist") else list(v),
        )
    elif vector_schema is not None:
        raise ValueError(
            f"VectorSchemaProvider only supported for ndarray. Got: {python_type}"
        )

    if isinstance(
        type_info.variant, (SequenceType, MappingType, RecordType, UnionType, AnyType)
    ):
        return _JSON_MAPPING

    return _JSON_MAPPING


# ============================================================

View on GitHub (pinned to e84aa99b32)

Solutions

  1. Pass the actual embedding dimension (positive integer), e.g. 384/768 depending on the model.
  2. Validate the size before constructing the provider: `if dim <= 0: raise ...` or assert at config load time.
  3. Fix the source of the dimension (model config, env var default) so it isn't 0.

Example fix

// before
provider = VectorSchemaProvider(size=len(embeds) if embeds else 0)
// after
dim = 768
assert dim > 0
provider = VectorSchemaProvider(size=dim)
Defensive patterns

Strategy: validation

Validate before calling

dim = get_embedding_dim(model)
if dim <= 0:
    raise ValueError(f"embedding dimension must be positive, got {dim}")
provider = VectorSchemaProvider(size=dim)

Type guard

def valid_vector_schema(s) -> bool:
    return s is not None and getattr(s, "size", 0) > 0

Try / catch

try:
    target = doris.declare_table(db, "tbl", record_type, vector_schema=provider)
except ValueError as e:
    if "Invalid vector dimension" in str(e):
        # fix the dimension source before retrying
        ...

Prevention

When it happens

Trigger: Constructing VectorSchemaProvider(size=0) (or negative) and passing it with an np.ndarray column to a Doris target declaration.

Common situations: Computing the embedding dimension from an uninitialized variable or an empty model output; typos like size=-1 as a sentinel; config where dimension comes from an env var defaulting to 0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/883e199517a6e6c9. Report an issue: GitHub.