pathwaycom/pathway · error · ValueError

vector column {name!r} has unsupported type {dtype}; a Pinec

Error message

vector column {name!r} has unsupported type {dtype}; a Pinecone vector must be a list[float] or a 1-D float array (dense), or a list[tuple[int, float]] of (index, weight) pairs (sparse).

What it means

pw.io.pinecone.write validates the vector column dtype at call time. Accepted shapes are a dense vector (list of numerics, 1-D numeric array, or numeric tuple) or a sparse vector (list[tuple[int, float]] of (index, weight) pairs). Anything else raises this ValueError listing the column and its dtype, mirroring the runtime InvalidVector/InvalidSparseVector guards.

Source

Thrown at python/pathway/io/pinecone/__init__.py:94

        raise ValueError(
            f"vector column {name!r} is nullable (type {dtype}); every row must "
            "carry a vector, so the column cannot be optional."
        )
    if isinstance(dtype, dt.List):
        inner = dtype.wrapped
        if _is_numeric(inner) or _is_sparse_pair(inner):
            return
        if isinstance(inner, (dt.List, dt.Array)):
            raise NotImplementedError(
                f"vector column {name!r} has type {dtype}, which is a multivector; "
                "a Pinecone record carries a single dense or sparse vector, so "
                "multivectors are not supported."
            )
    if isinstance(dtype, dt.Array) and _is_numeric(dtype.wrapped):
        return
    if isinstance(dtype, dt.Tuple) and all(_is_numeric(arg) for arg in dtype.args):
        return
    raise ValueError(
        f"vector column {name!r} has unsupported type {dtype}; a Pinecone vector "
        "must be a list[float] or a 1-D float array (dense), or a "
        "list[tuple[int, float]] of (index, weight) pairs (sparse)."
    )


def _check_metadata_dtype(name: str, dtype: dt.DType) -> None:
    """Reject a metadata column whose type Pinecone cannot store.

    Pinecone metadata supports ``int``, ``float``, ``bool``, ``str``, and
    ``list[str]``; ``None`` is allowed (it is dropped). Mirrors the runtime
    ``PineconeError::UnsupportedMetadataType`` guard.
    """
    inner = dtype.wrapped if isinstance(dtype, dt.Optional) else dtype
    if _is_statically_unknown(inner):
        return
    if inner in (dt.INT, dt.FLOAT, dt.BOOL, dt.STR):
        return

View on GitHub (pinned to fa2f74a464)

Solutions

Point the vector argument at the actual embedding column (list[float] or 1-D float array).
Parse serialized embeddings before the sink: json.loads per row or astype to produce list[float].
For sparse vectors, ensure the dtype is list[tuple[int, float]] — convert inner lists to tuples upstream.

Example fix

# before
pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.text)  # wrong column

# after
pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.embedding)  # list[float]

Defensive patterns

Strategy: type-guard

Validate before calling

import pathway as pw

def _is_numeric(d):
    return d in (pw.dt.INT, pw.dt.FLOAT, pw.dt.ANY) or isinstance(d, (pw.dt.Int, pw.dt.Float))

def is_dense_vector_dtype(dtype: pw.dt.DType) -> bool:
    return (isinstance(dtype, pw.dt.List) and _is_numeric(dtype.wrapped)) or (
        isinstance(dtype, pw.dt.Array) and _is_numeric(dtype.wrapped)
    )

assert is_dense_vector_dtype(table.schema[vector_col].dtype), "vector must be list[float] / 1-D float array"

Type guard

import pathway as pw

def is_sparse_vector_dtype(dtype: pw.dt.DType) -> bool:
    return (
        isinstance(dtype, pw.dt.List)
        and isinstance(dtype.wrapped, pw.dt.Tuple)
        and len(dtype.wrapped.args) == 2
        and dtype.wrapped.args[0] == pw.dt.INT
        and dtype.wrapped.args[1] == pw.dt.FLOAT
    )

Try / catch

try:
    pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.vec)
except ValueError as e:
    if "unsupported type" in str(e) and "vector" in str(e):
        docs = docs.with_columns(vec=docs.vec.apply(json.loads, return_type=list[float]))
        pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.vec)
    else:
        raise

Prevention

Always point the vector argument at the embedding column, never at raw text or metadata.
Parse stringified embeddings (JSON) into list[float] before the sink.
For sparse vectors, use list[tuple[int, float]], not nested lists.

When it happens

Trigger: Passing vector=table.col with dtype list[str], list[bool], a non-numeric array, a tuple mixing types, or any non-vector type to pw.io.pinecone.write.

Common situations: Pointing vector at the raw text column instead of the embedding column; embeddings serialized as strings (JSON) and not parsed; sparse vectors built as list[list[float]] instead of list[tuple[int, float]].

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/91dab9ea271c155e. Report an issue: GitHub.