pathwaycom/pathway · error · ValueError

primary_key column {name!r} has unsupported type {dtype}; a

Error message

primary_key column {name!r} has unsupported type {dtype}; a Pinecone record id must be int or str.

What it means

Pinecone record ids must be int or str (pointers are accepted because the engine stringifies them). pw.io.pinecone.write checks the primary_key column's dtype at call time and raises this ValueError for any other concrete type, mirroring the runtime PineconeError::InvalidId guard so a wrong column fails immediately rather than once data flows. Statically-unknown dtypes are not rejected here.

Source

Thrown at python/pathway/io/pinecone/__init__.py:49

def _check_primary_key_dtype(name: str, dtype: dt.DType) -> None:
    """Reject a ``primary_key`` whose type can never be a Pinecone record id.

    The id must always be present and must be an ``int`` or ``str`` (a pointer
    is accepted too, since the engine stringifies it). The matching runtime
    guard (``PineconeError::InvalidId``) only fires once an offending row reaches
    the sink, so catch the statically-known cases at ``write()`` time.
    """
    if _is_statically_unknown(dtype):
        return
    if isinstance(dtype, dt.Optional):
        raise ValueError(
            f"primary_key column {name!r} is nullable (type {dtype}); a Pinecone "
            "record id must always be present, so the column cannot be optional."
        )
    if dtype in (dt.INT, dt.STR) or isinstance(dtype, dt.Pointer):
        return
    raise ValueError(
        f"primary_key column {name!r} has unsupported type {dtype}; a Pinecone "
        "record id must be int or str."
    )


def _is_sparse_pair(dtype: dt.DType) -> bool:
    """Whether ``dtype`` is the ``tuple[int, float]`` of a sparse (index, weight) pair."""
    return (
        isinstance(dtype, dt.Tuple)
        and len(dtype.args) == 2
        and dtype.args[0] == dt.INT
        and dtype.args[1] == dt.FLOAT
    )


def _check_vector_dtype(name: str, dtype: dt.DType) -> None:
    """Reject a ``vector`` that is neither a dense nor a sparse vector column.

View on GitHub (pinned to fa2f74a464)

Solutions

1. Cast the id column to str before the sink, e.g. table.with_columns(id=table.id.astype(str)) or apply_windowed-free pw.this.transform.
2. Pick a column that is already int or str as the primary key.
3. If the id is a pointer column, that is fine as-is; otherwise convert floats/datetimes to their canonical string form.

Example fix

# before
pw.io.pinecone.write(docs, "idx", primary_key=docs.float_id, vector=docs.vec)

# after
docs = docs.with_columns(str_id=docs.float_id.astype(str))
pw.io.pinecone.write(docs, "idx", primary_key=docs.str_id, vector=docs.vec)

import pathway as pw

def ensure_pinecone_id_dtype(table, col_name: str):
    dtype = table.schema[col_name].dtype
    if dtype in (pw.dt.INT, pw.dt.STR) or isinstance(dtype, pw.dt.Pointer):
        return table
    return table.with_columns(**{col_name: table[col_name].astype(str)})

import pathway as pw

def is_pinecone_id_dtype(dtype: pw.dt.DType) -> bool:
    return dtype in (pw.dt.INT, pw.dt.STR) or isinstance(dtype, pw.dt.Pointer)

try:
    pw.io.pinecone.write(docs, "idx", primary_key=docs.float_id, vector=docs.vec)
except ValueError as e:
    if "unsupported type" in str(e) and "record id" in str(e):
        docs = docs.with_columns(id=docs.float_id.astype(str))
        pw.io.pinecone.write(docs, "idx", primary_key=docs.id, vector=docs.vec)
    else:
        raise

When it happens

Trigger: Passing primary_key=table.col where col has a non-int/str dtype, e.g. float, bool, datetime, or a tuple/list column, to pw.io.pinecone.write.

Common situations: Using a UUID column typed as anything other than str; pointing primary_key at a timestamp or numeric-measure column by mistake; ids stored as float from JSON ingestion.

Related errors

• primary_key column {name!r} is nullable (type {dtype}); a Pi
• metadata column {name!r} has unsupported type {dtype}; Pinec
• vector column {name!r} has type {dtype}, which is a multivec
• vector column {name!r} has unsupported type {dtype}; a Pinec
• invalid ssl mode '{ssl_mode}', expected one of disable, allo

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/a260615b87ea8d9d. Report an issue: GitHub.