pathwaycom/pathway · error · ValueError

column {col._name!r} is used as the {role} and cannot also b

Error message

column {col._name!r} is used as the {role} and cannot also be a metadata column. Remove it from metadata_columns.

What it means

Raised by pw.io.pinecone.write() when a column listed in metadata_columns is already used as the primary_key or the vector column. The connector would otherwise serialize the same column twice with conflicting roles, so it asks you to remove the duplicate from metadata_columns.

Source

Thrown at python/pathway/io/pinecone/__init__.py:378

    if metadata_columns is None:
        metadata_names = [
            col_name
            for col_name in table.column_names()
            if col_name not in (pk_name, vector_name)
        ]
    else:
        metadata_names = []
        for col in metadata_columns:
            if col._table is not table:
                raise ValueError(
                    f"metadata column {col._name!r} does not belong to the "
                    f"provided table. Pass column references from the same table, "
                    f"e.g. table.{col._name}."
                )
            if col._name in (pk_name, vector_name):
                role = "primary_key" if col._name == pk_name else "vector"
                raise ValueError(
                    f"column {col._name!r} is used as the {role} and cannot also "
                    "be a metadata column. Remove it from metadata_columns."
                )
            metadata_names.append(col._name)

    if batch_size <= 0:
        raise ValueError(f"batch_size must be a positive integer, got {batch_size}.")

    if pk_name is not None:
        _check_primary_key_dtype(pk_name, table._get_column(pk_name).dtype)
    _check_vector_dtype(vector_name, table._get_column(vector_name).dtype)
    for col_name in metadata_names:
        _check_metadata_dtype(col_name, table._get_column(col_name).dtype)

    resolved_api_key = (
        api_key if api_key is not None else os.environ.get("PINECONE_API_KEY")
    )
    if not resolved_api_key:

View on GitHub (pinned to fa2f74a464)

Solutions

Remove the pk/vector column from metadata_columns.
If you are in a hybrid setup writing two indexes from one table, spell out metadata_columns explicitly excluding both vector columns.

Example fix

# before
pw.io.pinecone.write(t, index_name="docs", primary_key=t.doc_id, vector=t.emb, metadata_columns=[t.doc_id, t.title], api_key=k)
# after
pw.io.pinecone.write(t, index_name="docs", primary_key=t.doc_id, vector=t.emb, metadata_columns=[t.title], api_key=k)

Defensive patterns

Strategy: validation

Validate before calling

reserved = {primary_key._name if primary_key is not None else None, vector._name}
metadata_columns = [c for c in metadata_columns if c._name not in reserved]

Prevention

When switching from default metadata to explicit metadata_columns, derive it: [table[c] for c in table.column_names() if c not in (pk, vec)].
Read the connector docstring section on hybrid retrieval, which warns the default would include the other vector column.

When it happens

Trigger: primary_key=table.doc_id together with metadata_columns=[table.doc_id, ...]; or vector=table.emb with metadata_columns containing table.emb.

Common situations: Migrating from a call that relied on the default metadata (all columns) to explicit metadata_columns and accidentally including the id or embedding column; the hybrid-search doc example warns the default would try to store the other vector column as metadata.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/39d5a65cb89c4822. Report an issue: GitHub.