pathwaycom/pathway · error · ValueError

Iceberg reader requires explicit primary key fields specific

Error message

Iceberg reader requires explicit primary key fields specification

What it means

The Pathway Iceberg reader (pw.io.iceberg.read) requires the input schema to declare a primary key, because Iceberg tables are upsert/read by identifier and the connector needs to know which column(s) form the key. It raises ValueError when schema.primary_key_columns() returns None, i.e. no column was marked with primary_key=True.

Source

Thrown at python/pathway/io/iceberg/__init__.py:189

    ...     name: str

    Then, this table must be read from the Iceberg storage.

    >>> input_table = pw.io.iceberg.read(
    ...     catalog=pw.io.iceberg.RestCatalog(uri="http://localhost:8181/"),
    ...     namespace=["app"],
    ...     table_name="users",
    ...     schema=InputSchema,
    ...     mode="static",
    ... )

    Don't forget to run your program with ``pw.run`` once you define all necessary
    computations. Note that you can also change the mode to ``"streaming"`` if you want
    the changes in the table to be reflected in your computational pipeline.
    """

    if schema.primary_key_columns() is None:
        raise ValueError(
            "Iceberg reader requires explicit primary key fields specification"
        )

    _check_entitlements("iceberg")
    schema, api_schema = read_schema(schema)

    data_storage = api.DataStorage(
        storage_type="iceberg",
        iceberg_catalog=catalog._to_engine(),
        table_name=table_name,
        namespace=namespace,
        mode=internal_connector_mode(mode),
    )
    data_format = api.DataFormat(
        format_type="transparent",
        **api_schema,
    )

View on GitHub (pinned to fa2f74a464)

Solutions

1. Declare the primary key in the schema: class InputSchema(pw.Schema): user_id: str = pw.column_definition(primary_key=True).
2. For composite keys, mark every key column with primary_key=True.
3. Verify the chosen column(s) match the actual Iceberg table's identifier fields before running pw.run.

Example fix

# before
class InputSchema(pw.Schema):
    user_id: str
    name: str
# after
class InputSchema(pw.Schema):
    user_id: str = pw.column_definition(primary_key=True)
    name: str

if InputSchema.primary_key_columns() is None:
    raise SystemExit(
        "Iceberg schema needs a primary_key=True column; "
        "see https://pathway.com/developers/user-guide/io/iceberg/"
    )

pw.io.iceberg.read(catalog=catalog, table_name="users", schema=InputSchema)

def iceberg_ready(schema) -> bool:
    return schema.primary_key_columns() is not None

When it happens

Trigger: pw.io.iceberg.read(catalog=..., namespace=['app'], table_name='users', schema=InputSchema) where InputSchema is a pw.Schema class in which no column uses pw.column_definition(primary_key=True).

Common situations: Defining the schema as a plain class without primary_key annotations; porting a schema from csv/json connectors where primary keys are optional; assuming the Iceberg table's key metadata is enough and not mirroring it in the Pathway schema.

Related errors

• Defining a primary key in the schema is not supported for pw
• pw.io.mssql.read requires at least one primary key column in
• pw.io.mssql.read primary_key column(s) {nullable_pks} are de
• pw.io.mysql.read requires at least one primary key column in
• pw.io.mysql.read primary_key column(s) {nullable_pks} are de

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/8f199feda2547b68. Report an issue: GitHub.