apache/beam · error · ValueError

invalid incremental, inc value must be within [%s, %s)

Error message

invalid incremental, inc value must be within [%s, %s)

What it means

_ObjectIdHelper.increment_id adds inc to the integer form of an ObjectId and requires the result to remain a valid 96-bit value: 0 <= id + inc < 2**96. Violating that raises ValueError('invalid incremental, inc value must be within [-id_number, 2**96 - id_number)'). Note the message itself computes the upper bound with an operator-precedence bug (1 << 96 - id_number), so the displayed upper bound can be misleading — the actual check uses new_number >= (1 << 96).

Source

Thrown at sdks/python/apache_beam/io/mongodbio.py:651

  def increment_id(
      cls,
      _id: ObjectId,
      inc: int,
  ) -> ObjectId:
    """
    Increment object_id binary value by inc value and return new object id.

    Args:
      _id: The `_id` to change.
      inc(int): The incremental int value to be added to `_id`.

    Returns:
        `_id` incremented by `inc` value
    """
    id_number = _ObjectIdHelper.id_to_int(_id)
    new_number = id_number + inc
    if new_number < 0 or new_number >= (1 << 96):
      raise ValueError(
          "invalid incremental, inc value must be within ["
          "%s, %s)" % (0 - id_number, 1 << 96 - id_number))
    return _ObjectIdHelper.int_to_id(new_number)


class WriteToMongoDB(PTransform):
  """WriteToMongoDB is a ``PTransform`` that writes a ``PCollection`` of
  mongodb document to the configured MongoDB server.

  In order to make the document writes idempotent so that the bundles are
  retry-able without creating duplicates, the PTransform added 2 transformations
  before final write stage:
  a ``GenerateId`` transform and a ``Reshuffle`` transform.::

                  -----------------------------------------------
    Pipeline -->  |GenerateId --> Reshuffle --> WriteToMongoSink|
                  -----------------------------------------------
                                  (WriteToMongoDB)

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure 0 <= id_to_int(_id) + inc < 2**96 before calling; clamp or wrap inc accordingly.
2. If you need a negative shift, verify id_number >= -inc first.
3. Reset to a fresh base ObjectId when increments approach the 96-bit limit.
4. Compute the step as inc = target_number - id_to_int(_id) from a valid target id.

Example fix

// before
next_id = _ObjectIdHelper.increment_id(_id, -10**30)
// after
n = _ObjectIdHelper.id_to_int(_id)
inc = max(-n, -10**30)  # clamp so result stays >= 0
next_id = _ObjectIdHelper.increment_id(_id, inc)

n = _ObjectIdHelper.id_to_int(_id)
if not (0 <= n + inc < (1 << 96)):
    raise ValueError('inc would push id out of 96-bit range')

def can_increment(_id, inc) -> bool:
    n = _ObjectIdHelper.id_to_int(_id)
    return 0 <= n + inc < (1 << 96)

try:
    new_id = _ObjectIdHelper.increment_id(_id, inc)
except ValueError:
    new_id = _ObjectIdHelper.int_to_id((n + inc) % (1 << 96))  # wrap

When it happens

Trigger: Calling _ObjectIdHelper.increment_id(_id, inc) where inc is negative with |inc| > id_number (underflow below 0) or so large that id_number + inc reaches 2**96 (overflow).

Common situations: Pagination/synthetic id generation that keeps incrementing past the maximum; passing a negative step larger than the current id; using increment sizes intended for 64-bit counters on a 96-bit ObjectId.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• number value must be within [0, %s)
• MongoDb location must be in the following format: 'mongodb:/
• Cannot create a filter for an unsupported node: ${node}
• Encountered an unexpected node kind: ${node.getKind()}
• Was expecting a RexCall or a boolean RexInputRef, but receiv

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1e08f86b4b5a0d1d. Report an issue: GitHub.