apache/beam · error · TypeError

Object of type ' ' is not JSON serializable

Error message

Object of type '%s' is not JSON serializable

What it means

Raised by the module's default_encoder when JsonDotSerializator/JSON serialization encounters an object it has no branch for (it handles dates, datetimes, times, Decimal, etc., but not arbitrary types). The encoder logs the offending object and raises TypeError to prevent silently producing invalid JSON for BigQuery API requests.

Solutions

1. Convert the offending value to a JSON-safe type before passing it (str(), float(), int(), isoformat()).
2. For Decimal/numpy values, cast explicitly: float(np_value) or str(decimal_value).
3. Encode bytes as base64 or a string before adding to the request.
4. Subclass the encoder and add a 'default' branch for your custom type if you own the serialization call site.

Example fix

# before
query_params = {'value': numpy.int64(5)}

# after
query_params = {'value': int(numpy.int64(5))}

import json
def ensure_json_safe(payload):
    json.dumps(payload)  # raises TypeError early for unserializable values

def is_json_safe(obj):
    try:
        json.dumps(obj)
        return True
    except (TypeError, ValueError):
        return False

try:
    send_request(payload)
except TypeError as e:
    if 'not JSON serializable' in str(e):
        payload = sanitize(payload)  # cast numpy/Decimal/bytes to str/int/float
    else:
        raise

When it happens

Trigger: Calling json.dumps(obj, cls=JsonDotSerializator/default_encoder) or any Beam code path that serializes a request payload (query config, job config) containing an object of an unhandled type, e.g. a numpy value, bytes, set, or custom class.

Common situations: Passing numpy types from dataframe pipelines into query parameters; putting bytes or enums into job labels/parameters; inserting datetime subclasses or timezone objects not covered by the isoformat branch.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• . . Row: %r
• Cannot convert to a JSON value.
• Error writing row to Avro
• Failed to convert the row to JSON
• Failed to convert the row to JSON

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4d33028298753179. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/bigquery_tools.py:162

  DEFLATE = 'DEFLATE'
  SNAPPY = 'SNAPPY'
  NONE = 'NONE'


def default_encoder(obj):
  if isinstance(obj, decimal.Decimal):
    return str(obj)
  elif isinstance(obj, bytes):
    # on python 3 base64-encoded bytes are decoded to strings
    # before being sent to BigQuery
    return obj.decode('utf-8')
  elif isinstance(obj, (datetime.date, datetime.time)):
    return str(obj)
  elif isinstance(obj, datetime.datetime):
    return obj.isoformat()

  _LOGGER.error("Unable to serialize %r to JSON", obj)
  raise TypeError(
      "Object of type '%s' is not JSON serializable" % type(obj).__name__)


def get_hashable_destination(destination):
  """Parses a table reference into a (project, dataset, table) tuple.

  Args:
    destination: Either a TableReference object from the bigquery API.
      The object has the following attributes: projectId, datasetId, and
      tableId. Or a string representing the destination containing
      'PROJECT:DATASET.TABLE'.
  Returns:
    A string representing the destination containing
    'PROJECT:DATASET.TABLE'.
  """
  if isinstance(destination, TableReference):
    return '%s:%s.%s' % (
        destination.projectId, destination.datasetId, destination.tableId)

View on GitHub (pinned to 12126d8942)