pathwaycom/pathway · error · ValueError

The column should be of the type 'bytes'

Error message

The column should be of the type 'bytes'

What it means

Raised by pw.io.pubsub.write when the table's single column has a dtype other than dt.BYTES or dt.ANY. Pub/Sub message payloads are opaque bytes, so str/int/float/bool/JSON-typed columns are rejected at setup time; dt.ANY is tolerated because its runtime type is checked later in the callback.

Source

Thrown at python/pathway/io/pubsub/__init__.py:133

    >>> topic = publisher.create_topic(request={"name": topic_path})  # doctest: +SKIP

    After that you can configure the table output with the following code:

    >>> import pathway as pw
    >>> pw.io.pubsub.write(table, publisher, project_id, topic_id)  # doctest: +SKIP

    At last, don't forget to add ``pw.run()`` to run your pipeline.
    """

    columns = list(table._columns.values())
    if len(columns) != 1:
        raise ValueError(
            f"Unexpected number of columns in table: {len(table._columns)}"
        )

    allowed_column_types = (dt.BYTES, dt.ANY)
    if columns[0].dtype not in allowed_column_types:
        raise ValueError("The column should be of the type 'bytes'")

    output_buffer = _OutputBuffer(publisher, project_id, topic_id)
    subscribe(
        table,
        on_change=output_buffer.on_change,
        on_time_end=output_buffer.on_time_end,
        name=name,
        sort_by=sort_by,
    )

View on GitHub (pinned to fa2f74a464)

Solutions

1. Convert the column to bytes before writing: encode strings with an apply/astype into a bytes column, or declare it as bytes in the schema and encode at ingest.
2. For string payloads, encode explicitly at the source (e.g. .encode('utf-8') before the value enters the table).
3. If the type genuinely varies, type the column as dt.ANY and ensure runtime values are bytes (the runtime check will then enforce it).

Example fix

# before
class S(pw.Schema):
    data: str
pw.io.pubsub.write(t, publisher, project_id, topic_id)

# after
class S(pw.Schema):
    data: bytes
pw.io.pubsub.write(t, publisher, project_id, topic_id)

from pathway import dt
col = next(iter(t._columns.values()))
assert col.dtype in (dt.BYTES, dt.ANY), f"payload column must be bytes, got {col.dtype}"
pw.io.pubsub.write(t, publisher, project_id, topic_id)

def payload_is_bytes_compatible(t) -> bool:
    from pathway import dt
    col = next(iter(t._columns.values()))
    return col.dtype in (dt.BYTES, dt.ANY)

When it happens

Trigger: pw.io.pubsub.write(t, ...) where the single column is typed str, int, float, or any non-BYTES dtype — e.g. t.select(data=t.some_str_column) or a schema declaring value: str.

Common situations: Publishing text payloads without encoding to bytes first; passing a JSON-string column (dtype str) instead of a bytes column; schemas inferred from CSV/JSON where strings map to dt.STRING.

Related errors

• Unexpected value type. Expected bytes, got {type(data)}
• Unexpected number of columns: {len(row)}
• Unexpected number of columns in table: {len(table._columns)}
• demo.noisy_linear_stream error: nb_rows should be strictly p
• Pathway does not support using binary operator {expression._

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/5363ea32be260429. Report an issue: GitHub.