pathwaycom/pathway · error · ValueError

primary_key column {primary_key._name!r} does not belong to

Error message

primary_key column {primary_key._name!r} does not belong to the provided table. Pass a column reference from the same table, e.g. primary_key=table.{primary_key._name}.

What it means

The Weaviate output connector requires that the primary_key column reference belongs to the same pw.Table instance passed as `table`; the key is encoded into Weaviate's deterministic object UUID, so a column from a different table would silently index the wrong data. write() compares primary_key._table to table and raises this ValueError with a suggested corrected expression when they differ.

Source

Thrown at python/pathway/io/weaviate/__init__.py:116

    ...     DocSchema,
    ...     [(1, [0.1, 0.2, 0.3, 0.4]), (2, [0.5, 0.6, 0.7, 0.8])],
    ... )

    Attach the Weaviate output connector, mapping the primary key and the vector
    column:

    >>> pw.io.weaviate.write(  # doctest: +SKIP
    ...     table,
    ...     collection_name="Docs",
    ...     primary_key=table.doc_id,
    ...     vector=table.embedding,
    ... )
    >>> pw.run(monitoring_level=pw.MonitoringLevel.NONE)  # doctest: +SKIP
    """
    _check_entitlements("weaviate")

    if primary_key is not None and primary_key._table is not table:
        raise ValueError(
            f"primary_key column {primary_key._name!r} does not belong to the "
            f"provided table. Pass a column reference from the same table, "
            f"e.g. primary_key=table.{primary_key._name}."
        )
    if vector is not None and vector._table is not table:
        raise ValueError(
            f"vector column {vector._name!r} does not belong to the provided "
            f"table. Pass a column reference from the same table, "
            f"e.g. vector=table.{vector._name}."
        )

    pk = primary_key._name if primary_key is not None else None
    vector_field = vector._name if vector is not None else None

    # Weaviate reserves "id" and "vector" as object-level keys and rejects them as
    # property names. The primary key (encoded in the UUID) and the vector column
    # are never sent as properties, so they may freely use these names; any other
    # column that does collides and is reported up front.

View on GitHub (pinned to fa2f74a464)

Solutions

Pass the column reference from the very table object you pass as `table`: primary_key=table.doc_id as the message suggests.
Re-derive the key column on the transformed table before writing (e.g. keep the key through filters/joins so table.doc_id exists).

Example fix

# before
t_full = pw.io.csv.read(...)
t = t_full.filter(pw.this.visible)
pw.io.weaviate.write(t, collection_name="Docs", primary_key=t_full.doc_id)

# after
pw.io.weaviate.write(t, collection_name="Docs", primary_key=t.doc_id)

Defensive patterns

Strategy: validation

Validate before calling

def check_column_belongs(table, col, arg="primary_key"):
    if col is not None and col._table is not table:
        raise ValueError(f"{arg} column {col.name!r} is not from the given table")
    return True

Prevention

Always reference columns from the exact object you pass as `table`; re-derive references after every filter/join/select since they return new objects.
Write connector calls immediately after the final transformation, keeping only one table variable in scope.

When it happens

Trigger: Calling pw.io.weaviate.write(table_a, ..., primary_key=table_b.doc_id) where table_b is any table other than table_a — e.g. a filtered, joined, or otherwise derived table object.

Common situations: Passing a column captured before a table transformation (t_filtered = t.filter(...) creates a new object); reusing a variable from an earlier pipeline step; passing schema.col class-attribute style references from a different table.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/90f844549d276a7c. Report an issue: GitHub.