apache/beam · error · ValueError

primary_key_field is required when action='IGNORE'

Error message

primary_key_field is required when action='IGNORE'

What it means

ConflictResolution.__post_init__ enforces that when action='IGNORE' a primary_key_field is supplied, because IGNORE-strategy deduplicates conflicting rows by matching the primary key. Without it the ignore semantics cannot be implemented, so configuration fails at construction.

Solutions

Set primary_key_field to the table's primary key column name when using action='IGNORE'.
Verify the field name matches a column in your column_specs/table.
If you actually want upsert semantics, use action='UPDATE' with update_fields instead.

Example fix

// before
ConflictResolution(action="IGNORE")
// after
ConflictResolution(action="IGNORE", primary_key_field="id")

Defensive patterns

Strategy: validation

Validate before calling

if cr.action == 'IGNORE' and not cr.primary_key_field:
    raise ValueError("primary_key_field required for IGNORE conflict resolution")

Type guard

def ignore_is_valid(cr) -> bool:
    return cr.action != "IGNORE" or cr.primary_key_field is not None

Try / catch

try:
    cr = ConflictResolution(action="IGNORE", **kw)
except ValueError as e:
    if "primary_key_field is required" in str(e):
        cr = ConflictResolution(action="IGNORE", primary_key_field="id", **kw)
    else:
        raise

Prevention

Pair every action='IGNORE' with primary_key_field in your config templates
Validate conflict-resolution config at pipeline startup
Document per-action required fields (UPDATE->update_fields, IGNORE->primary_key_field)

When it happens

Trigger: Constructing ConflictResolution(action='IGNORE') without primary_key_field — either omitted or left as its default None.

Common situations: Switching action from 'UPDATE' (which needs update_fields, not primary_key_field) to 'IGNORE' without adding the key field; building the config from YAML/JSON where the primary key entry is missing.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/62ad0c47e4593775. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/rag/ingestion/mysql_common.py:433

        >>> ConflictResolution(
        ...     action="IGNORE", 
        ...     primary_key_field="id"
        ... )
        
        Ignore conflicts with custom primary key:
        >>> ConflictResolution(
        ...     action="IGNORE",
        ...     primary_key_field="custom_id"
        ... )
    """
  action: Literal["UPDATE", "IGNORE"] = "UPDATE"
  update_fields: Optional[list[str]] = None
  primary_key_field: Optional[str] = None

  def __post_init__(self):
    """Validate configuration after initialization."""
    if self.action == "IGNORE" and self.primary_key_field is None:
      raise ValueError("primary_key_field is required when action='IGNORE'")

View on GitHub (pinned to 12126d8942)