apache/beam · error · ValueError

Unexpected outputs from validation

Error message

Unexpected outputs from validation: {list(validation_result.keys())}

What it means

The Validate transform used for output_schema checking is expected to return only the main (good) output and, if configured, an error output. If validation_result still contains other keys after integrating the error output, _integrate_validation_results raises this ValueError — an internal invariant that the validation produced unexpected extra outputs.

Solutions

1. Ensure the validation step only produces the main output and the configured error output.
2. If using a custom Validate transform, drop or merge extra outputs before returning.
3. Check the Beam version — this can indicate an internal incompatibility; upgrade/downgrade apache-beam to matching versions.
4. File an issue with the Beam YAML team if a built-in Validate produces this.

Example fix

// before (custom Validate)
def expand(self, pcoll):
    return {'good': ok, 'bad': errors, 'stats': stats}
// after
def expand(self, pcoll):
    return {'output': ok, self._error_tag: errors}

def check_validate_outputs(result_keys, error_tag=None):
    allowed = {'output', error_tag} - {None}
    extra = set(result_keys) - allowed
    if extra:
        raise ValueError(f'Validate produced unexpected outputs: {extra}')

def only_expected_keys(result: dict, allowed: set) -> bool:
    return set(result) <= allowed

try:
    expand_output_schema_transform(spec, outputs, eh)
except ValueError as e:
    if 'Unexpected outputs from validation' in str(e):
        print('Validation transform emitted extra outputs; pin apache-beam version or fix custom Validate')
    else:
        raise

When it happens

Trigger: A Validate transform (or custom validation implementation) emits additional tagged outputs beyond the good/error outputs, e.g. three or more outputs or unexpected tag names, while output_schema integration expects exactly zero leftover keys.

Common situations: Custom Validate implementations or third-party validation transforms that add extra side outputs; version drift where a validation transform gained new outputs not handled by the YAML integration code.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• Edge source and target cannot be empty
• HuggingFacePipelineModelHandler requires either 'task' or…
• Incompatible types: vs
• Missing config in ML transform spec
• Missing output in error_handling of

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8c49ef346f99be26. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:730

    outputs[main_output_key] = validation_result
    return outputs

  # The main output from validation is the good output.
  main_tag = error_handling_spec.get('main_tag', 'good')
  outputs[main_output_key] = validation_result.pop(main_tag)

  if error_handling_spec:
    error_output_tag = error_handling_spec['output']
    if error_output_tag in validation_result:
      schema_error_pcoll = validation_result.pop(error_output_tag)
      # The original transform also had an error output. Merge them.
      outputs[error_output_tag] = (
          (outputs[error_output_tag], schema_error_pcoll)
          | f'FlattenErrors_{main_output_key}' >> beam.Flatten())

    # There should be no other outputs from validation.
    if validation_result:
      raise ValueError(
          "Unexpected outputs from validation: "
          f"{list(validation_result.keys())}")

  return outputs


def _enforce_schema(pcoll, label, error_handling_spec, clean_schema):
  """Applies schema to PCollection elements if necessary, then validates.

  This function ensures that the input PCollection conforms to a specified
  schema. If the PCollection is schemaless (i.e., its element_type is Any),
  it attempts to convert its elements into schema-aware `beam.Row` objects
  based on the provided `clean_schema`. After ensuring the PCollection has
  a defined schema, it applies a `Validate` transform to perform the actual
  schema validation.

  Args:
    pcoll: The input PCollection to be schema-enforced and validated.

View on GitHub (pinned to 12126d8942)