pathwaycom/pathway · error · ValueError

Column {pseudocolumn} has to contain integers only.

Error message

Column {pseudocolumn} has to contain integers only.

What it means

Pathway encodes start_from="end" as the sentinel -1 in the start_from_timestamp_ms field, so user-supplied timestamps must be non-negative; a negative value would silently be interpreted as 'start from the end of the stream'. The validator rejects any timestamp < 0 to preserve that sentinel contract.

Source

Thrown at python/pathway/debug/__init__.py:305

    series_dict = {}
    for name in columns:
        dtype = _dtype_to_pandas(table.schema.typehints()[name])
        if include_id:
            series = pd.Series(columns[name], dtype=dtype)
        else:
            # we need to remove original keys, otherwise pandas will use them to create index
            series = pd.Series(list(columns[name].values()), dtype=dtype)
        series_dict[name] = series
    index = keys if include_id else None
    res = pd.DataFrame(series_dict, index=index)
    return res


def _validate_dataframe(df: pd.DataFrame, stacklevel: int = 1) -> None:
    for pseudocolumn in api.PANDAS_PSEUDOCOLUMNS:
        if pseudocolumn in df.columns:
            if not pd.api.types.is_integer_dtype(df[pseudocolumn].dtype):
                raise ValueError(f"Column {pseudocolumn} has to contain integers only.")
    if api.TIME_PSEUDOCOLUMN in df.columns:
        if any(df[api.TIME_PSEUDOCOLUMN] < 0):
            raise ValueError(
                f"Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times."
            )
        if any(df[api.TIME_PSEUDOCOLUMN] % 2 == 1):
            warn(
                "timestamps are required to be even; all timestamps will be doubled",
                stacklevel=stacklevel + 1,
            )
            df[api.TIME_PSEUDOCOLUMN] = 2 * df[api.TIME_PSEUDOCOLUMN]

    if api.DIFF_PSEUDOCOLUMN in df.columns:
        if any((df[api.DIFF_PSEUDOCOLUMN] != 1) & (df[api.DIFF_PSEUDOCOLUMN] != -1)):
            raise ValueError(
                f"Column {api.DIFF_PSEUDOCOLUMN} can only have 1 and -1 values."
            )

View on GitHub (pinned to fa2f74a464)

Solutions

1. Use a non-negative epoch-milliseconds value, e.g. start_from_timestamp_ms=0 to start from the very beginning of the retained stream.
2. If the intent was 'latest', use start_from="end" and remove the timestamp argument.
3. Check how the timestamp is computed (e.g. datetime deltas) and fix the sign/units before passing it in.

Example fix

# before
pw.io.kafka.read(..., start_from="timestamp", start_from_timestamp_ms=-1)

# after
pw.io.kafka.read(..., start_from="timestamp", start_from_timestamp_ms=0)

from datetime import datetime, timezone

ts_ms = int(start_dt.timestamp() * 1000)
assert ts_ms >= 0, f"timestamp must be >= 0, got {ts_ms}"
pw.io.kafka.read(..., start_from="timestamp", start_from_timestamp_ms=ts_ms)

def is_valid_timestamp_ms(ts: int | None) -> bool:
    return ts is not None and ts >= 0

When it happens

Trigger: Calling a Pathway IO read API with start_from="timestamp" and a negative start_from_timestamp_ms, e.g. start_from_timestamp_ms=-1 or any value below zero.

Common situations: Developer uses -1 (or 0 minus something) as a 'replay everything' flag, mimicking other systems; or computes a timestamp from a negative date arithmetic result (e.g. subtraction overflow producing a negative ms value).

Related errors

• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• Unsupported argument for {data_format!r} format: 'value'
• 'delimiter' is only meaningful for the 'dsv' format, but {fo
• 'subject' was provided without 'schema_registry_settings'. T

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/dddb656a3e4ce6ae. Report an issue: GitHub.