apache/beam · error · ValueError

Converting %r to datetime truncates it to microseconds. Set…

Error message

Converting %r to datetime truncates it to microseconds. Set allow_lossy_conversion=True to allow this conversion.

What it means

Timestamp.to_utc_datetime() converts to a Python datetime, which can only hold microsecond resolution. If the Timestamp has nanosecond precision, the conversion truncates digits, so the method raises ValueError unless allow_lossy_conversion=True explicitly permits that truncation.

Solutions

Call ts.to_utc_datetime(allow_lossy_conversion=True) to accept truncation to microseconds.
Convert to microsecond precision first with to_precision(6, allow_lossy_conversion=True) for a single explicit truncation point.
Use ts.to_rfc3339() with lossy flag or work with `.nanos` if full precision must be preserved.
Intercept ValueError around to_utc_datetime and implement custom rounding/formatting that keeps nanos.

Example fix

// before
dt = ts.to_utc_datetime()
// after
dt = ts.to_utc_datetime(allow_lossy_conversion=True)

Defensive patterns

Strategy: validation

Validate before calling

def safe_to_datetime(ts, allow_lossy=False):
    if ts.precision > 6 and not allow_lossy:
        raise ValueError('nanosecond precision would be truncated')
    return ts.to_utc_datetime(allow_lossy_conversion=allow_lossy)

Type guard

def fits_in_datetime(ts):
    return ts.precision <= 6

Try / catch

try:
    dt = ts.to_utc_datetime()
except ValueError:
    dt = ts.to_precision(6, allow_lossy_conversion=True).to_utc_datetime()

Prevention

Remember Python datetime only supports microsecond resolution.
Keep nanosecond data in Beam Timestamps; convert to datetime only for display.
Pass allow_lossy_conversion=True deliberately where truncation is acceptable.

When it happens

Trigger: Calling ts.to_utc_datetime() (directly or via _utc, to_language_type, or to_rfc3339) on a Timestamp with _precision > 6 without allow_lossy_conversion=True.

Common situations: Formatting nanosecond timestamps for logging or JSON output via to_rfc3339; converting beam Timestamps to datetime for use with Python datetime APIs after ingesting nanosecond data.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d8e41813721d7da1. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/utils/timestamp.py:319

    avoid offset due to default timezone mismatch.

    Args:
      has_tz: whether the timezone info is attached, default to False.
      allow_lossy_conversion: must be set to True to convert a timestamp
        with precision above microseconds, since ``datetime.datetime`` only
        supports microsecond resolution; the result is truncated (floored)
        to whole microseconds.

    Returns:
      a ``datetime.datetime`` object of UTC for this Timestamp.

    Raises:
      ValueError: if this timestamp has precision above microseconds and
        allow_lossy_conversion is not True.
    """
    if self._precision > Timestamp.MICROS_PRECISION:
      if not allow_lossy_conversion:
        raise ValueError(
            'Converting %r to datetime truncates it to microseconds. Set '
            'allow_lossy_conversion=True to allow this conversion.' % self)
      micros_of_second = self._subseconds // _POW_10[self._precision -
                                                     Timestamp.MICROS_PRECISION]
    else:
      micros_of_second = self._subseconds * _POW_10[Timestamp.MICROS_PRECISION -
                                                    self._precision]
    # We can't easily construct a datetime object from microseconds, so we
    # create one at the epoch and add an appropriate timedelta interval.
    epoch = self._epoch_datetime_utc()
    if not has_tz:
      epoch = epoch.replace(tzinfo=None)
    return epoch + datetime.timedelta(
        seconds=self._seconds, microseconds=micros_of_second)

  def to_rfc3339(self) -> str:
    """Returns an RFC 3339 string for this Timestamp."""
    if self._precision <= Timestamp.MICROS_PRECISION:

View on GitHub (pinned to 12126d8942)