apache/beam · error · ValueError

. . Row: %r

Error message

%s. %s. Row: %r

What it means

Raised by `JsonCoder.encode` (RowAsDictJsonCoder) when json.dumps fails — typically because the row dict contains NaN or Infinity (allow_nan=False) — wrapping the original ValueError with the JSON_COMPLIANCE_ERROR explanation and the offending row. BigQuery streaming rows must be valid JSON, and NaN/Infinity literals are not JSON-compliant.

Solutions

1. Sanitize the row before writing: replace NaN/inf with None (NULL column) or a sentinel numeric value.
2. Use math.isnan/math.isinf checks in a map step before the BigQuery sink.
3. In pandas/numpy sources, apply df.replace([np.inf, -np.inf], np.nan).where(df.notna(), None).
4. Fix the computation producing NaN/inf (guard divisions, clip log/exp inputs).

Example fix

// before
rows | beam.io.WriteToBigQuery(table, method=beam.io.WriteToBigQuery.Method.FILE_LOADS)
// after
def clean(row):
    return {k: (None if isinstance(v, float) and (math.isnan(v) or math.isinf(v)) else v) for k, v in row.items()}
rows | beam.Map(clean) | beam.io.WriteToBigQuery(table, method=beam.io.WriteToBigQuery.Method.FILE_LOADS)

import math
def json_safe_row(row):
    return {k: (None if isinstance(v, float) and (math.isnan(v) or math.isinf(v)) else v)
            for k, v in row.items()}
# map rows through this before the BigQuery sink

def is_json_compliant_value(v):
    return not (isinstance(v, float) and (math.isnan(v) or math.isinf(v)))

try:
    encoded = coder.encode(row)
except ValueError as e:
    if 'JSON_COMPLIANCE' in str(e) or 'Out of range' in str(e) or 'NaN' in str(e):
        row = sanitize_row(row)
        encoded = coder.encode(row)
    else:
        raise

When it happens

Trigger: Encoding a row dict containing float('nan'), float('inf'), or -inf (common from 0/0, numpy operations, or missing-value placeholders) when writing to BigQuery with the file_loads or streaming-insert JSON path.

Common situations: Pipelines over numpy/pandas data where NaN is a default missing marker; computed metrics dividing by zero; ML feature pipelines emitting inf from log/exp transforms; upstream CSV loads using NaN placeholders.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• Object of type ' ' is not JSON serializable
• Cannot convert to a JSON value.
• Error writing row to Avro
• Failed to convert the row to JSON
• Failed to convert the row to JSON

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f51908436393001c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/bigquery_tools.py:1449

class RowAsDictJsonCoder(coders.Coder):
  """A coder for a table row (represented as a dict) to/from a JSON string.

  This is the default coder for sources and sinks if the coder argument is not
  specified.
  """
  def encode(self, table_row):
    # The normal error when dumping NAN/INF values is:
    # ValueError: Out of range float values are not JSON compliant
    # This code will catch this error to emit an error that explains
    # to the programmer that they have used NAN/INF values.
    try:
      return json.dumps(
          table_row,
          allow_nan=False,
          ensure_ascii=False,
          default=default_encoder).encode('utf-8')
    except ValueError as e:
      raise ValueError(
          '%s. %s. Row: %r' % (e, JSON_COMPLIANCE_ERROR, table_row))

  def decode(self, encoded_table_row):
    return json.loads(encoded_table_row.decode('utf-8'))

  def to_type_hint(self):
    return Any


class JsonRowWriter(io.IOBase):
  """
  A writer which provides an IOBase-like interface for writing table rows
  (represented as dicts) as newline-delimited JSON strings.
  """
  def __init__(self, file_handle):
    """Initialize an JsonRowWriter.

    Args:

View on GitHub (pinned to 12126d8942)