provided transform_name_mapping without setting the --update

Error message

provided transform_name_mapping without setting the --update flag, so the pipeline would not be updated

What it means

Setting --transform_name_mapping only has an effect when updating an existing job. getJobOptions rejects a mapping provided without --update, because the mapping would be ignored and the pipeline would run as a new job instead of an update — silently breaking the user's intent.

Source

Thrown at sdks/go/pkg/beam/runners/dataflow/dataflow.go:313

	if *autoscalingAlgorithm != "" {
		if *autoscalingAlgorithm != "NONE" && *autoscalingAlgorithm != "THROUGHPUT_BASED" {
			return nil, errors.New("invalid autoscaling algorithm. Use --autoscaling_algorithm=(NONE|THROUGHPUT_BASED)")
		}
	}

	if *flexRSGoal != "" {
		switch *flexRSGoal {
		case "FLEXRS_UNSPECIFIED", "FLEXRS_SPEED_OPTIMIZED", "FLEXRS_COST_OPTIMIZED":
			// valid values
		default:
			return nil, errors.Errorf("invalid flex resource scheduling goal. Got %q; Use --flexrs_goal=(FLEXRS_UNSPECIFIED|FLEXRS_SPEED_OPTIMIZED|FLEXRS_COST_OPTIMIZED)", *flexRSGoal)
		}
	}
	if !streaming && *transformMapping != "" {
		return nil, errors.New("provided transform_name_mapping for a batch pipeline, did you mean to construct a streaming pipeline?")
	}
	if !*update && *transformMapping != "" {
		return nil, errors.New("provided transform_name_mapping without setting the --update flag, so the pipeline would not be updated")
	}
	var updateTransformMapping map[string]string
	if *transformMapping != "" {
		if err := json.Unmarshal([]byte(*transformMapping), &updateTransformMapping); err != nil {
			return nil, errors.Wrapf(err, "error reading --transform_name_mapping flag as JSON")
		}
	}
	if *usePublicIPs == *noUsePublicIPs {
		useSet := isFlagPassed("use_public_ips")
		noUseSet := isFlagPassed("no_use_public_ips")
		// If use_public_ips was explicitly set but no_use_public_ips was not, use that value
		// We take the explicit value of no_use_public_ips if it was set but use_public_ips was not.
		if useSet && !noUseSet {
			*noUsePublicIPs = !*usePublicIPs
		} else if useSet && noUseSet {
			return nil, errors.New("exactly one of usePublicIPs and noUsePublicIPs must be true, please check that only one is true")
		}
	}

View on GitHub (pinned to 12126d8942)

Solutions

Add --update to the pipeline invocation when a transform_name_mapping is supplied
Remove --transform_name_mapping if a fresh job (not an update) was intended
Gate the mapping flag together with the update flag in launch scripts

Example fix

// before
--transform_name_mapping='{"old":"new"}'
// after
--update --transform_name_mapping='{"old":"new"}'

Defensive patterns

Strategy: validation

Validate before calling

if transformMapping != "" && !update {
    return errors.New("--update is required with --transform_name_mapping")
}

Prevention

Always pair --update with --transform_name_mapping
Gate both flags together in launch scripts

When it happens

Trigger: Passing --transform_name_mapping=<json> without --update on a streaming pipeline.

Common situations: Forgetting --update when re-running a job with renamed transforms intending an in-place update; scripts where the update flag is toggled by condition but the mapping flag is always set.

Understand the failure class

Background: "--flag is required" and "must specify" CLI errors: how missing-required-flag validation works and how to fix it — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/cfcfbe66a0f40eb8. Report an issue: GitHub.