pathwaycom/pathway · error · ValueError

negative timestamp cannot be used

Error message

negative timestamp cannot be used

What it means

pw.io.airbyte.read validates the connector's configured catalog before running it: every stream must have sync_mode set to either "incremental" or "full_refresh". Any other value in the catalog JSON is rejected up front.

Source

Thrown at python/pathway/debug/__init__.py:548

        is a mapping from worker id to list of rows processed in this batch by this worker,
        and each row is tuple (diff, key, values).

        Note: unless you need to specify timestamps and keys, consider using
        `table_from_list_of_batches` and `table_from_list_of_batches_by_workers`.

        Args:
            batches: dictionary with specified batches to be put in the table
            schema: schema of the table
        """
        unique_name = self._get_next_unique_name()
        workers = {worker for batch in batches.values() for worker in batch}
        for worker in workers:
            self.events[(unique_name, worker)] = []

        timestamps = set(batches.keys())

        if any(timestamp for timestamp in timestamps if timestamp < 0):
            raise ValueError("negative timestamp cannot be used")
        elif any(timestamp for timestamp in timestamps if timestamp == 0):
            warn(
                "rows with timestamp 0 are only backfilled and are not processed by output connectors"
            )

        if any(timestamp for timestamp in timestamps if timestamp % 2 == 1):
            warn(
                "timestamps are required to be even; all timestamps will be doubled",
                stacklevel=_stacklevel + 1,
            )
            batches = {2 * timestamp: batches[timestamp] for timestamp in batches}

        for timestamp in sorted(batches):
            self._advance_time_for_all_workers(unique_name, workers, timestamp)
            batch = batches[timestamp]
            for worker, changes in batch.items():
                for diff, key, values in changes:
                    if diff == 1:

View on GitHub (pinned to fa2f74a464)

Solutions

1. Set each stream's sync_mode to "incremental" or "full_refresh".
2. If copying from an Airbyte catalog export, use the source's sync_mode field, not the destination's destination_sync_mode.
3. Let Pathway configure streams automatically (e.g. streams="*") instead of hand-writing the catalog.

Example fix

# before
streams=[{"stream": "users", "sync_mode": "append"}]

# after
streams=[{"stream": "users", "sync_mode": "incremental"}]

ALLOWED = {"incremental", "full_refresh"}
for s in streams:
    mode = s["sync_mode"] if isinstance(s, dict) else s.sync_mode
    assert mode in ALLOWED, f"bad sync_mode {mode!r} for stream {s}"
pw.io.airbyte.read(..., streams=streams)

from typing import Any

def valid_stream_list(streams: Any) -> bool:
    if not isinstance(streams, list):
        return True  # e.g. "*" is fine
    return all(
        s.get("sync_mode") in {"incremental", "full_refresh"}
        for s in streams if isinstance(s, dict)
    )

When it happens

Trigger: Passing a streams list (or catalog dict) to pw.io.airbyte.read where a stream's sync_mode field is misspelled or unsupported, e.g. {"stream": "users", "sync_mode": "full_refresh"} vs a value like "incremental_" or "append".

Common situations: Hand-writing the streams argument from the Airbyte spec and using a destination sync mode (e.g. "append", "overwrite") instead of the source sync mode; or a catalog JSON exported from Airbyte containing an empty/renamed sync_mode.

Related errors

• demo.range_stream error: nb_rows should be strictly positive
• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• schema does not match given dataframe
• only diffs of 1 and -1 are supported

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/eddba9f055468c36. Report an issue: GitHub.