apache/beam · error · argparse.ArgumentError

The maximum allowed number of GCS custom audit entries…

Error message

The maximum allowed number of GCS custom audit entries (including the default x-goo-custom-audit-job) is %d.

What it means

The GCS custom audit entries argparse action enforces a maximum number of entries, counting the always-present default x-goo-custom-audit-job entry. Raising more entries than MAX_ENTRIES triggers this error so the job doesn't get rejected later by the service.

Solutions

Reduce the number of custom audit entries to at most MAX_ENTRIES - 1
Merge related metadata into fewer, shorter entries
Keep the most important entries and move the rest to a separate tracking system

Example fix

// before
--gcs_custom_audit_entries=k1=v1 k2=v2 k3=v3 k4=v4 k5=v5 ...
// after
--gcs_custom_audit_entries=k1=v1 k2=v2

Defensive patterns

Strategy: validation

Validate before calling

entries = [f'{k}={v}' for k, v in audit_map.items()]
assert len(entries) <= _GcsCustomAuditEntriesAction.MAX_ENTRIES - 1, 'too many audit entries'

Type guard

def within_entry_limit(entries):
    return len(entries) < _GcsCustomAuditEntriesAction.MAX_ENTRIES

Try / catch

try:
    parse_args(argv)
except argparse.ArgumentError as ex:
    print(f'too many audit entries: {ex}')
    sys.exit(2)

Prevention

Cap the audit map at build time (e.g. dict slice)
Merge metadata into fewer entries
Test pipeline construction with the maximum allowed entries

When it happens

Trigger: Supplying more than _GcsCustomAuditEntriesAction.MAX_ENTRIES - 1 custom audit entries via the gcs_custom_audit_entries pipeline option (repeated key=value arguments).

Common situations: Bulk-attaching many audit labels to satisfy compliance tooling; generating audit entries in a loop from a config map without a cap.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5c8c4f61229faa65. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/options/pipeline_options.py:224

      setattr(namespace, self.dest, {})
      self._custom_audit_entries = getattr(namespace, self.dest)

    if option_string == '--gcs_custom_audit_entries':
      # in the format of {"key": "value"}
      assert (isinstance(values, str))
      sub_entries = json.loads(values)
      for key, value in sub_entries.items():
        self._add_entry(key, value)
    else:  # option_string == '--gcs_custom_audit_entry'
      # in the format of 'key=value'
      assert (isinstance(values, str))
      parts = values.split('=', 1)
      key = parts[0]
      value = parts[1] if len(parts) > 1 else ''
      self._add_entry(key, value)

    if self._exceed_entry_limit():
      raise argparse.ArgumentError(
          None,
          "The maximum allowed number of GCS custom audit entries (including the default x-goo-custom-audit-job) is %d."  # pylint: disable=line-too-long
          % _GcsCustomAuditEntriesAction.MAX_ENTRIES)


class _CommaSeparatedListAction(argparse.Action):
  """
  Argparse Action that splits comma-separated values and appends them to
  a list. This allows options like --experiments=abc,def to be treated
  as separate experiments 'abc' and 'def', similar to how Java SDK handles
  them.

  If there are key=value experiments in a raw argument, the remaining part of
  the argument are treated as values and won't split further. For example:
  'abc,def,master_key=k1=v1,k2=v2' becomes
  ['abc', 'def', 'master_key=k1=v1,k2=v2'].
  """
  def __call__(self, parser, namespace, values, option_string=None):

View on GitHub (pinned to 12126d8942)