apache/beam · error · TypeError

Can not encode {} as a 64-bit integer

Error message

Can not encode {} as a 64-bit integer

What it means

to_json_value encodes Python ints into the extra_types.JsonValue integer_value field, which is a signed 64-bit integer. Values outside [_MININT64, _MAXINT64] (about +/-9.22e18) raise TypeError 'Can not encode {} as a 64-bit integer'. Python ints are arbitrary precision, but the proto field is not.

Source

Thrown at sdks/python/apache_beam/internal/gcp/json_value.py:114

    json_object = extra_types.JsonObject()
    for k, v in obj.items():
      json_object.properties.append(
          extra_types.JsonObject.Property(
              key=k, value=to_json_value(v, with_type=with_type)))
    return extra_types.JsonValue(object_value=json_object)
  elif with_type:
    return to_json_value(get_typed_value_descriptor(obj), with_type=False)
  elif isinstance(obj, str):
    return extra_types.JsonValue(string_value=obj)
  elif isinstance(obj, bytes):
    return extra_types.JsonValue(string_value=obj.decode('utf8'))
  elif isinstance(obj, bool):
    return extra_types.JsonValue(boolean_value=obj)
  elif isinstance(obj, int):
    if _MININT64 <= obj <= _MAXINT64:
      return extra_types.JsonValue(integer_value=obj)
    else:
      raise TypeError('Can not encode {} as a 64-bit integer'.format(obj))
  elif isinstance(obj, float):
    return extra_types.JsonValue(double_value=obj)
  elif isinstance(obj, ValueProvider):
    if obj.is_accessible():
      return to_json_value(obj.get())
    return extra_types.JsonValue(is_null=True)
  else:
    raise TypeError('Cannot convert %s to a JSON value.' % repr(obj))


def from_json_value(v):
  """For internal use only; no backwards-compatibility guarantees.

  Converts ``extra_types.JsonValue`` objects into Python objects.

  Args:
    v: ``JsonValue`` object to be converted.

View on GitHub (pinned to 12126d8942)

Solutions

1. Encode as a string: to_json_value(str(bigint)) and parse back after transport.
2. Convert to float if approximate magnitude suffices (to_json_value(float(obj))), accepting precision loss.
3. Clamp or split the value at the application level to fit int64.
4. Keep the value out of the JSON-encoded payload and pass it via a different channel.

Example fix

// before
to_json_value(1 << 70)  # TypeError: not a 64-bit integer
// after
to_json_value(str(1 << 70))  # encode as string, parse on receipt

_MININT64, _MAXINT64 = -(2**63), 2**63 - 1
def is_int64_safe(v) -> bool:
    return not isinstance(v, int) or isinstance(v, bool) or _MININT64 <= v <= _MAXINT64

def fits_int64(v) -> bool:
    return isinstance(v, bool) or (isinstance(v, int) and -(2**63) <= v <= 2**63 - 1)

try:
    return to_json_value(v)
except TypeError as e:
    if '64-bit integer' in str(e):
        return to_json_value(str(v))  # encode big ints as strings
    raise

When it happens

Trigger: to_json_value on an int larger than 2^63-1 or smaller than -2^63 — huge counters, cryptographic-size integers, bit-manipulation results (1 << 64), or oversized IDs; also triggered recursively when encoding containers/ValueProviders holding such ints.

Common situations: Passing arbitrary-precision Python ints (hashes, big random numbers, snowflake-style IDs wider than 64 bits) into Beam APIs that JSON-encode values; shifting bits beyond 63; summing counters that overflow 64-bit range.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• {element}
• Cannot get a type descriptor for %s.
• %s. %s
• Only strings allowd as map keys when converting from JSON, f
• Missing required field: {name}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/11a6c75234de621f. Report an issue: GitHub.