apache/beam · error · TypeError

Cannot convert to a JSON value.

Error message

Cannot convert %s to a JSON value.

What it means

apache_beam.internal.gcp.json_value.to_json_value converts Python objects (str, bytes, int, float, bool, dict, list, proto types, ValueProviders) into extra_types.JsonValue protobuf wrappers. When the input object is none of the supported types, the function raises TypeError with the repr of the offending object. It exists because JsonValue is a oneof-wrapped protobuf and cannot hold arbitrary Python objects.

Solutions

1. Convert the object to a supported type first (str, int, float, bool, dict, list, or None) before calling to_json_value
2. For datetime objects, serialize explicitly with .isoformat() or .timestamp()
3. For numpy scalars, cast with .item() or int()/float()
4. If passing a ValueProvider, ensure it is accessible (is_accessible() True) or accept the null default
5. Wrap the call in try/except TypeError and provide a custom serialization for your type

Example fix

// before
from apache_beam.internal.gcp.json_value import to_json_value
import datetime
v = to_json_value(datetime.date(2024, 1, 1))  # TypeError
// after
v = to_json_value(datetime.date(2024, 1, 1).isoformat())

def is_json_convertible(obj):
    return obj is None or isinstance(obj, (str, bytes, bool, int, float, dict, list))

def is_json_convertible(obj):
    return obj is None or isinstance(obj, (str, bytes, bool, int, float, dict, list))

try:
    jv = to_json_value(obj)
except TypeError as e:
    jv = to_json_value(str(obj))  # or custom serializer

When it happens

Trigger: Calling to_json_value(obj) where obj is not str, bytes, bool, int, float, None, dict/list of convertible values, a recognized protobuf type (StructuredValue, Row, LogicalType), or an accessible ValueProvider — e.g. passing a set, datetime, numpy scalar, or custom class.

Common situations: Users pass datetime.date/numpy.int64 values from pipelines into BigQuery helper paths that serialize via JsonValue; passing an inaccessible or exotic object type; internal Beam code receiving unexpectedly typed config values.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• Cannot convert from a JSON value.
• Cannot get a type descriptor for
• No SerializeToString method is detected on loaded model…
• Object of type ' ' is not JSON serializable
• . . Row: %r

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/81186d2ccc227ca0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/internal/gcp/json_value.py:122

  elif isinstance(obj, str):
    return extra_types.JsonValue(string_value=obj)
  elif isinstance(obj, bytes):
    return extra_types.JsonValue(string_value=obj.decode('utf8'))
  elif isinstance(obj, bool):
    return extra_types.JsonValue(boolean_value=obj)
  elif isinstance(obj, int):
    if _MININT64 <= obj <= _MAXINT64:
      return extra_types.JsonValue(integer_value=obj)
    else:
      raise TypeError('Can not encode {} as a 64-bit integer'.format(obj))
  elif isinstance(obj, float):
    return extra_types.JsonValue(double_value=obj)
  elif isinstance(obj, ValueProvider):
    if obj.is_accessible():
      return to_json_value(obj.get())
    return extra_types.JsonValue(is_null=True)
  else:
    raise TypeError('Cannot convert %s to a JSON value.' % repr(obj))


def from_json_value(v):
  """For internal use only; no backwards-compatibility guarantees.

  Converts ``extra_types.JsonValue`` objects into Python objects.

  Args:
    v: ``JsonValue`` object to be converted.

  Returns:
    A Python object structured as values, lists, and dictionaries corresponding
    to ``JsonValue``, ``JsonArray`` and ``JsonObject`` types.

  Raises:
    TypeError: if the ``JsonValue`` object contains a type that is
      not supported.

View on GitHub (pinned to 12126d8942)