pathwaycom/pathway · error · ValueError

The 'max_difference' must either be an integer or a datetime

Error message

The 'max_difference' must either be an integer or a datetime.timedelta

What it means

ValueError raised by the connector synchronization API when max_difference is neither an int nor a datetime.timedelta. Ints are interpreted as seconds; timedeltas express the allowed inter-source time skew directly.

Source

Thrown at python/pathway/io/_synchronization.py:239

    - Later, the first source sends a record with ``T + 1h``. \
      This record is not yet forwarded for processing because it exceeds ``max_difference``.
    - If the second source then sends a record with ``T + 1h``, the system detects a 1-hour gap. \
      Since both sources have moved beyond ``T``, the synchronization group accepts ``T + 1h`` \
      as the new baseline and continues processing from there.
    - However, if the second source instead sends a record with ``T + 5m``, this record \
      is processed normally. The system will continue waiting for the first source to \
      catch up before advancing further.

    This behavior ensures that data gaps do not cause deadlocks but are properly detected and handled.
    """

    if len(columns) < 2:
        raise ValueError("At least two columns must participate in a connector group")

    if not isinstance(max_difference, int) and not isinstance(
        max_difference, datetime.timedelta
    ):
        raise ValueError(
            "The 'max_difference' must either be an integer or a datetime.timedelta"
        )

    if isinstance(max_difference, int) and max_difference < 0:
        raise ValueError("The 'max_difference' can't be negative")
    if isinstance(
        max_difference, datetime.timedelta
    ) and max_difference < datetime.timedelta(0):
        raise ValueError("The 'max_difference' can't be negative")

    column_types = set()
    for synchronized_column in columns:
        if isinstance(synchronized_column, ColumnReference):
            column = synchronized_column
            priority = SynchronizedColumn.__dataclass_fields__["priority"].default
            idle_duration = None
        else:
            column = synchronized_column.column

View on GitHub (pinned to fa2f74a464)

Solutions

Use an int (seconds): max_difference=600
Or a datetime.timedelta: max_difference=datetime.timedelta(minutes=10)
Coerce config values before calling: int(cfg['max_difference']) or datetime.timedelta(seconds=float(cfg['max_difference']))

Example fix

// before
pw.io.synchronize(a.time, b.time, max_difference='600')

// after
pw.io.synchronize(a.time, b.time, max_difference=600)  # or datetime.timedelta(seconds=600)

Defensive patterns

Strategy: validation

Validate before calling

import datetime

def coerce_max_difference(v):
    if isinstance(v, datetime.timedelta):
        return v
    if isinstance(v, int) and not isinstance(v, bool):
        return v
    if isinstance(v, str):
        return int(v)  # or raise, per project policy
    raise TypeError('max_difference must be int seconds or datetime.timedelta')

Type guard

import datetime

def is_valid_max_difference(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) or isinstance(v, datetime.timedelta)

Prevention

Normalize max_difference to int seconds or timedelta at config load time
Never pass strings or floats straight from YAML/env into the sync API
Unit-test config parsing for synchronization parameters

When it happens

Trigger: Passing max_difference='600' (string), 600.0 (float), pandas.Timedelta, or datetime.datetime to the sync/synchronize function. Note: bool is technically an int subclass but a float seconds value is rejected.

Common situations: Reading max_difference from YAML/env as a string; using pandas.Timedelta from a config library; passing milliseconds as float; copy-pasting a timedelta expression that yields a Date.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/ff8ac4a2f7a9a47d. Report an issue: GitHub.