apache/beam · error · ValueError

HuggingFacePipelineModelHandler requires either 'task' or 'm

Error message

HuggingFacePipelineModelHandler requires either 'task' or 'model' to be specified.

What it means

Static validation for the HuggingFace pipeline handler requires the YAML config to name at least one of 'task' (e.g. text-classification) or 'model' (hub id/path). If the config is empty or lacks both keys, validate() raises this ValueError.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_ml.py:369

    handler_kwargs = {}
    if inference_fn_obj:
      handler_kwargs['inference_fn'] = inference_fn_obj

    _handler = HuggingFacePipelineModelHandler(
        task=task,
        model=model,
        device=device,
        load_pipeline_args=load_pipeline_args,
        **handler_kwargs,
        **kwargs)

    super().__init__(_handler, preprocess, postprocess)

  @staticmethod
  def validate(config):
    if not config or (not config.get('task') and not config.get('model')):
      raise ValueError(
          "HuggingFacePipelineModelHandler requires either 'task' or "
          "'model' to be specified.")

  def inference_output_type(self):
    return Any


@beam.ptransform.ptransform_fn
def run_inference(
    pcoll,
    model_handler: dict[str, Any],
    inference_tag: Optional[str] = 'inference',
    inference_args: Optional[dict[str, Any]] = None) -> beam.PCollection[beam.Row]:  # pylint: disable=line-too-long
  """
  A transform that takes the input rows, containing examples (or features), for
  use on an ML model. The transform then appends the inferences
  (or predictions) for those examples to the input row.

View on GitHub (pinned to 12126d8942)

Solutions

1. Add 'task' to the handler config, e.g. task: text-classification
2. Or add 'model' with a HuggingFace model id or local path
3. Check YAML indentation so task/model sit inside config
4. Fix key misspellings to exactly 'task' or 'model'

Example fix

# before
- type: HuggingFacePipeline
  config: {}
# after
- type: HuggingFacePipeline
  config:
    task: text-classification

def validate_hf_config(config):
    if not isinstance(config, dict) or not (config.get('task') or config.get('model')):
        raise ValueError("HuggingFace handler config needs 'task' or 'model'")

def is_valid_hf_config(config):
    return isinstance(config, dict) and bool(config.get('task') or config.get('model'))

When it happens

Trigger: Calling HuggingFacePipelineModelHandler.validate with a config dict that is None/empty or contains neither 'task' nor 'model', e.g. YAML config: {} or config with only preprocess/postprocess.

Common situations: Omitting the config block entirely in a YAML pipeline; misspelling the key (e.g. 'model_id' or 'task_name'); nesting the values under the wrong level of YAML indentation.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Node ID cannot be empty
• Edge source and target cannot be empty
• Incompatible types: {weak_schema['type']} vs {strong_schema[
• Unknown output name "{tag}" from {by}
• Unexpected parameters in model_handler: {extra_params}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8f84b584a91381f5. Report an issue: GitHub.