apache/beam · error · ValueError

%s. %s

Error message

%s. %s

What it means

When encoding a TableRow to JSON, json.dumps raised ValueError (e.g. NaN/Infinity values, since allow_nan=False). The error is re-raised with the original message plus a note that BigQuery JSON must be compliant (no NaN/Infinity).

Source

Thrown at sdks/python/apache_beam/io/gcp/bigquery.py:575

    if self.table_schema:
      self.field_names = tuple(fs.name for fs in self.table_schema.fields)
      self.field_types = tuple(fs.type for fs in self.table_schema.fields)

  def encode(self, table_row):
    if self.table_schema is None:
      raise AttributeError(
          'The TableRowJsonCoder requires a table schema for '
          'encoding operations. Please specify a table_schema argument.')
    try:
      return json.dumps(
          collections.OrderedDict(
              zip(
                  self.field_names,
                  [from_json_value(f.v) for f in table_row.f])),
          allow_nan=False,
          default=bigquery_tools.default_encoder)
    except ValueError as e:
      raise ValueError('%s. %s' % (e, bigquery_tools.JSON_COMPLIANCE_ERROR))

  def decode(self, encoded_table_row):
    od = json.loads(
        encoded_table_row, object_pairs_hook=collections.OrderedDict)
    return bigquery.TableRow(
        f=[bigquery.TableCell(v=to_json_value(e)) for e in od.values()])


class BigQueryDisposition(object):
  """Class holding standard strings used for create and write dispositions."""

  CREATE_NEVER = 'CREATE_NEVER'
  CREATE_IF_NEEDED = 'CREATE_IF_NEEDED'
  WRITE_TRUNCATE = 'WRITE_TRUNCATE'
  WRITE_APPEND = 'WRITE_APPEND'
  WRITE_EMPTY = 'WRITE_EMPTY'

  @staticmethod

View on GitHub (pinned to 12126d8942)

Solutions

1. Clean the data before writing: replace NaN/inf with None (beam.Map with math.isfinite check)
2. Use to_json_value from bigquery_tools or convert non-compliant floats to strings
3. Set a CoGroup/filter step to drop or fix offending records

Example fix

// before
# row.f contains float('nan')
encoded = coder.encode(row)
// after
import math
clean = [None if isinstance(v, float) and not math.isfinite(v) else v for v in values]
row = TableRow(f=[TableCell(v=v) for v in clean])

import math
def row_is_json_safe(row):
    return all(not isinstance(c.v, float) or math.isfinite(c.v) for c in row.f)
assert row_is_json_safe(row), 'row contains NaN/Infinity'

def is_json_safe(v):
    return not (isinstance(v, float) and not math.isfinite(v))

try:
    encoded = coder.encode(row)
except ValueError as e:
    if 'JSON_COMPLIANCE' in str(e) or 'Out of range float' in str(e):
        row = sanitize_row(row)  # map non-finite floats to None
        encoded = coder.encode(row)
    else:
        raise

When it happens

Trigger: Encoding a TableRow whose cell values contain NaN, Infinity, or otherwise non-JSON-compliant values, in TableRowJsonCoder.encode.

Common situations: Pipeline data computed with floats producing NaN/inf (division by zero, missing fills) that is then written to BigQuery as JSON rows.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• Please specify a BigQuery table to read from.
• Encountered unsupported parameter(s) in read_gbq: {kwargs.ke
• Cannot get a type descriptor for %s.
• Can not encode {} as a 64-bit integer
• %s can be set with either schema_update_options or additiona

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4f6323d4704d498a. Report an issue: GitHub.