pathwaycom/pathway · error · ValueError

primary_key column {primary_key._name!r} does not belong to

Error message

primary_key column {primary_key._name!r} does not belong to the provided table. Pass a column reference from the same table, e.g. primary_key=table.{primary_key._name}.

What it means

The primary_key argument of pw.io.milvus.write must be a ColumnReference bound to the exact table being written. Pathway identity-checks the reference's table and raises ValueError (with a suggested correct form table.<name>) when the reference comes from a different Table instance.

Source

Thrown at python/pathway/io/milvus/__init__.py:292

    Milvus primary key field:

    >>> pw.io.milvus.write(   # doctest: +SKIP
    ...     table,
    ...     uri="./milvus.db",
    ...     collection_name="docs",
    ...     primary_key=table.doc_id,
    ... )
    >>> pw.run(monitoring_level=pw.MonitoringLevel.NONE)  # doctest: +SKIP
    """
    _check_entitlements("milvusdb")
    with optional_imports("milvus"):
        from pymilvus import MilvusClient

    if batch_size < 1:
        raise ValueError(f"batch_size must be a positive integer, got {batch_size}.")

    if primary_key._table is not table:
        raise ValueError(
            f"primary_key column {primary_key._name!r} does not belong to the "
            f"provided table. Pass a column reference from the same table, "
            f"e.g. primary_key=table.{primary_key._name}."
        )

    client = _make_client(MilvusClient, uri)

    # Fail fast if the collection is missing: otherwise the error would only
    # surface deep inside pw.run() on the first upsert, or — for an empty table —
    # never, silently running a misconfigured pipeline that writes nothing.
    if not client.has_collection(collection_name):
        client.close()
        raise ValueError(
            f"Milvus collection {collection_name!r} does not exist; create it "
            f"before writing. pw.io.milvus.write never creates a collection "
            f"because it cannot infer the vector field's dimension."
        )

View on GitHub (pinned to fa2f74a464)

Solutions

Pass the final table and its own column together: pw.io.milvus.write(final, ..., primary_key=final.doc_id)
Select the key into the written table if it lives elsewhere: final = t.join(keys, ...).select(..., doc_id=keys.doc_id)
Reference by name defensively: table[table.schema.primary_key_columns()[0]] when wiring dynamically

Example fix

# before
ready = raw.select(id=raw.doc_id, vec=embed(raw.text))
pw.io.milvus.write(raw, uri, "docs", primary_key=ready.id)

# after
ready = raw.select(id=raw.doc_id, vec=embed(raw.text))
pw.io.milvus.write(ready, uri, "docs", primary_key=ready.id)

Defensive patterns

Strategy: validation

Validate before calling

assert primary_key._table is table, (
    f"primary_key {primary_key._name!r} not from the written table; "
    f"use table.{primary_key._name}"
)

Type guard

def pk_from_table(table: pw.Table, col: pw.ColumnReference) -> bool:
    return col._table is table

Try / catch

try:
    pw.io.milvus.write(table, uri, "docs", primary_key=primary_key)
except ValueError as e:
    if "does not belong to the provided table" in str(e):
        primary_key = table[primary_key._name]
        pw.io.milvus.write(table, uri, "docs", primary_key=primary_key)
    else:
        raise

Prevention

Select the primary key into the final table before wiring the sink
Pass primary_key=final.pk with the same final table variable
Use table[name] lookups in generic connector wrappers

When it happens

Trigger: pw.io.milvus.write(base, ..., primary_key=derived.doc_id) where derived is any transform of base (filter/select/with_columns all create new Table objects).

Common situations: Reusing a key reference captured before transforms while passing the transformed table, or vice versa; primary keys taken from a join partner table.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/9e092e2e6fe82693. Report an issue: GitHub.