pathwaycom/pathway · error · ValueError

metadata column {name!r} has unsupported type {dtype}; Pinec

Error message

metadata column {name!r} has unsupported type {dtype}; Pinecone metadata supports int, float, bool, str, and list[str].

What it means

Pinecone metadata only supports int, float, bool, str, list[str], and tuples of str (None is dropped). pw.io.pinecone.write checks each metadata column's dtype at call time and raises this ValueError for unsupported types (dicts, nested lists, datetimes, etc.), mirroring the runtime PineconeError::UnsupportedMetadataType guard. The Optional wrapper is unwrapped first, and statically-unknown inner types are allowed through.

Source

Thrown at python/pathway/io/pinecone/__init__.py:121

    Pinecone metadata supports ``int``, ``float``, ``bool``, ``str``, and
    ``list[str]``; ``None`` is allowed (it is dropped). Mirrors the runtime
    ``PineconeError::UnsupportedMetadataType`` guard.
    """
    inner = dtype.wrapped if isinstance(dtype, dt.Optional) else dtype
    if _is_statically_unknown(inner):
        return
    if inner in (dt.INT, dt.FLOAT, dt.BOOL, dt.STR):
        return
    if isinstance(inner, dt.List) and (
        inner.wrapped == dt.STR or _is_statically_unknown(inner.wrapped)
    ):
        return
    if isinstance(inner, dt.Tuple) and all(
        arg == dt.STR or _is_statically_unknown(arg) for arg in inner.args
    ):
        return
    raise ValueError(
        f"metadata column {name!r} has unsupported type {dtype}; Pinecone "
        "metadata supports int, float, bool, str, and list[str]."
    )


@check_arg_types
@trace_user_frame
def write(
    table: Table,
    index_name: str,
    *,
    primary_key: ColumnReference | None = None,
    vector: ColumnReference,
    api_key: str | None = None,
    host: str | None = None,
    namespace: str = "",
    metadata_columns: Iterable[ColumnReference] | None = None,
    batch_size: int = 100,

View on GitHub (pinned to fa2f74a464)

Solutions

1. Serialize unsupported columns before the sink: dates via .dt.strftime()/to_string, dicts/objects via json.dumps into a str column.
2. Cast list[int] metadata to list[str] if the numbers are labels, or move it out of metadata_columns.
3. Only list columns whose dtype is one of int, float, bool, str, or list[str] in metadata_columns.

Example fix

# before
pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.vec,
                    metadata_columns=[docs.payload, docs.created_at])
# payload: dict, created_at: datetime -> rejected

# after
docs = docs.with_columns(
    payload_str=docs.payload.apply(lambda d: json.dumps(d), return_type=str),
    created_at_str=docs.created_at.dt.strftime("%Y-%m-%dT%H:%M:%S"),
)
pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.vec,
                    metadata_columns=[docs.payload_str, docs.created_at_str])

import pathway as pw

ALLOWED = (pw.dt.INT, pw.dt.FLOAT, pw.dt.BOOL, pw.dt.STR)

def unsupported_metadata_columns(schema) -> list[str]:
    bad = []
    for name in schema.column_names():
        d = schema[name].dtype
        if isinstance(d, pw.dt.Optional):
            d = d.wrapped
        ok = d in ALLOWED or (isinstance(d, pw.dt.List) and d.wrapped == pw.dt.STR)
        if not ok:
            bad.append(name)
    return bad

assert not unsupported_metadata_columns(docs.schema), "serialize these columns before metadata_columns"

import pathway as pw

def is_pinecone_metadata_dtype(dtype: pw.dt.DType) -> bool:
    if isinstance(dtype, pw.dt.Optional):
        dtype = dtype.wrapped
    if dtype in (pw.dt.INT, pw.dt.FLOAT, pw.dt.BOOL, pw.dt.STR):
        return True
    return isinstance(dtype, pw.dt.List) and dtype.wrapped == pw.dt.STR

try:
    pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.vec,
                        metadata_columns=[docs.payload, docs.ts])
except ValueError as e:
    if "metadata" in str(e) and "unsupported type" in str(e):
        docs = docs.with_columns(
            payload=docs.payload.apply(json.dumps, return_type=str),
            ts=docs.ts.dt.strftime("%Y-%m-%dT%H:%M:%S"),
        )
        pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.vec,
                            metadata_columns=[docs.payload, docs.ts])
    else:
        raise

When it happens

Trigger: Including a metadata column with dtype dict[str, str], list[int], datetime, bytes, or nested structures in the metadata_columns argument of pw.io.pinecone.write.

Common situations: Passing raw JSON/payload columns as metadata; dict-typed columns from JSON ingestion; timestamp columns expecting Pinecone to accept them (it only accepts str — serialize first).

Related errors

• primary_key column {name!r} has unsupported type {dtype}; a
• primary_key column {name!r} is nullable (type {dtype}); a Pi
• vector column {name!r} has type {dtype}, which is a multivec
• vector column {name!r} has unsupported type {dtype}; a Pinec
• metadata column {col._name!r} does not belong to the provide

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/381cde5dea9aa60e. Report an issue: GitHub.