cocoindex-io/cocoindex · error · ValueError

Invalid pgvector dimension

Error message

Invalid pgvector dimension: {vector_schema.size}

What it means

Validation in the PostgreSQL target's type mapping. An np.ndarray column maps to pgvector, whose dimension must be known and positive; it is taken from the VectorSchema provided via an Annotated NDArray or column_overrides. This ValueError fires when the declared vector size is invalid (zero/negative/missing), since no well-typed vector column can be created. Declare the field with a VectorSchema carrying a positive dimension.

Solutions

  1. Set the provider size to the actual embedding dimension (e.g. 384, 768, 1536).
  2. If size is computed dynamically, assert it is > 0 before constructing the target.
  3. Check the provider implementation for an off-by-default/uninitialized size value.

Example fix

// before
overrides = {"embedding": VectorSchemaProvider(size=0)}
// after
overrides = {"embedding": VectorSchemaProvider(size=768)}
Defensive patterns

Strategy: validation

Validate before calling

dim = get_vector_size()
assert isinstance(dim, int) and dim > 0, f"pgvector dim must be positive, got {dim}"

Type guard

def valid_vector_dim(provider) -> bool:
    size = provider.size
    return isinstance(size, int) and size > 0

Try / catch

try:
    target = await PgTableTarget.from_class(Row, primary_key=["id"], column_overrides=ov)
except ValueError as e:
    if "Invalid pgvector dimension" in str(e):
        ov["embedding"] = VectorSchemaProvider(size=EMBEDDING_DIM)

Prevention

When it happens

Trigger: Supplying a VectorSchemaProvider whose size is 0 or negative, e.g. VectorSchemaProvider(size=0) or a provider computing size from an empty/uninitialized array.

Common situations: Computing dimension from an embedding model output before it has run, copy-paste of a placeholder size, or dynamic sizing that evaluates to 0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/24bb042c0482d8b7. Report an issue: GitHub.

Appendix: source

Thrown at python/cocoindex/connectors/postgres/_target.py:289

    type_info = analyze_type_info(python_type)

    # Check for PgType annotation override
    for annotation in type_info.annotations:
        if isinstance(annotation, PgType):
            return _TypeMapping(annotation.pg_type, annotation.encoder)

    base_type = type_info.base_type

    # Check direct leaf type mappings
    if base_type in _LEAF_TYPE_MAPPINGS:
        return _LEAF_TYPE_MAPPINGS[base_type]

    # NumPy ndarray: map to pgvector type bases; dimension is handled at the schema layer.
    if base_type is np.ndarray:
        if vector_schema is None:
            raise ValueError("VectorSpecProvider is required for NumPy ndarray type.")
        if vector_schema.size <= 0:
            raise ValueError(f"Invalid pgvector dimension: {vector_schema.size}")

        # Default to `vector` (float32/float64/int64/etc.). Use `halfvec` for float16.
        base = "halfvec" if vector_schema.dtype in (np.half, np.float16) else "vector"
        return _TypeMapping(
            pg_type=f"{base}({vector_schema.size})", encoder=_vector_encoder
        )

    elif vector_schema is not None:
        raise ValueError(
            f"VectorSpecProvider is only supported for NumPy ndarray type. Got type: {python_type}"
        )

    # Complex types that need JSON encoding
    if isinstance(
        type_info.variant, (SequenceType, MappingType, RecordType, UnionType, AnyType)
    ):
        return _JSONB_MAPPING

View on GitHub (pinned to e84aa99b32)