apache/beam · error · ValueError

'item_bytes' must be a non-negative number

Error message

'item_bytes' must be a non-negative number

What it means

ByteLimitedQueue tracks capacity in bytes, so put() requires item_bytes to be a non-negative size estimate of the queued item. A negative size would corrupt the queue's byte accounting, so Beam raises ValueError before enqueuing.

Source

Thrown at sdks/python/apache_beam/utils/byte_limited_queue.py:82

    If the queue is full, block until a free slot is available, unless `block`
    is false or a timeout occurs.

    Args:
      item: The item to put into the queue.
      item_bytes: The size of the item.
      block: If True, block until space is available. If False, raise queue.Full
        immediately if the queue is full.
      timeout: If block is True, wait for at most `timeout` seconds. If None,
        block indefinitely.

    Raises:
      ValueError: If timeout or item_bytes is negative.
      queue.Full: If the queue is full and block is False or the timeout occurs.
    """
    if timeout is not None and timeout < 0:
      raise ValueError("'timeout' must be a non-negative number")
    if item_bytes < 0:
      raise ValueError("'item_bytes' must be a non-negative number")

    with self._mutex:
      if not self._waiting_writers and self._can_fit(item_bytes):
        self._queue.append((item, item_bytes))
        self._byte_size += item_bytes
        self._not_empty.notify()
        return

      if not block:
        raise queue.Full

      # Reuse or create a condition
      my_cond = (
          self._condition_pool.pop()
          if self._condition_pool else threading.Condition(self._mutex))

      endtime = time.monotonic() + timeout if timeout is not None else None

View on GitHub (pinned to 12126d8942)

Solutions

  1. Ensure the computed size is non-negative: item_bytes = max(0, computed_size).
  2. Fix the size calculation (e.g. sys.getsizeof / len of encoded bytes).
  3. Use item_bytes=0 for zero-cost sentinel items rather than negatives.
  4. Add an assertion on computed sizes before calling put.

Example fix

// before
q.put(data, item_bytes=len(data) - header_len)
// after
q.put(data, item_bytes=max(0, len(data) - header_len))
Defensive patterns

Strategy: validation

Validate before calling

if item_bytes < 0:
    raise ValueError('item_bytes must be >= 0')

Try / catch

try:
    q.put(item, item_bytes=size)
except ValueError:
    q.put(item, item_bytes=max(0, size))

Prevention

When it happens

Trigger: Calling byte_queue.put(item, item_bytes=-1), or passing a size computed as len(serialized) - offset where offset exceeded length.

Common situations: Encoding size computation bugs (negative deltas); uninitialized size variables defaulting oddly; off-by-one errors in record-size bookkeeping.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ead1d533029513e1. Report an issue: GitHub.