error reading --transform_name_mapping flag as JSON

Error message

error reading --transform_name_mapping flag as JSON

What it means

getJobOptions parses the --transform_name_mapping flag as a JSON object of transform name mappings used with pipeline update. If json.Unmarshal fails, the raw parse error is wrapped with this message and returned. The flag is only honored when --update is set, and must be a valid JSON string-to-string object.

Source

Thrown at sdks/go/pkg/beam/runners/dataflow/dataflow.go:318

	if *flexRSGoal != "" {
		switch *flexRSGoal {
		case "FLEXRS_UNSPECIFIED", "FLEXRS_SPEED_OPTIMIZED", "FLEXRS_COST_OPTIMIZED":
			// valid values
		default:
			return nil, errors.Errorf("invalid flex resource scheduling goal. Got %q; Use --flexrs_goal=(FLEXRS_UNSPECIFIED|FLEXRS_SPEED_OPTIMIZED|FLEXRS_COST_OPTIMIZED)", *flexRSGoal)
		}
	}
	if !streaming && *transformMapping != "" {
		return nil, errors.New("provided transform_name_mapping for a batch pipeline, did you mean to construct a streaming pipeline?")
	}
	if !*update && *transformMapping != "" {
		return nil, errors.New("provided transform_name_mapping without setting the --update flag, so the pipeline would not be updated")
	}
	var updateTransformMapping map[string]string
	if *transformMapping != "" {
		if err := json.Unmarshal([]byte(*transformMapping), &updateTransformMapping); err != nil {
			return nil, errors.Wrapf(err, "error reading --transform_name_mapping flag as JSON")
		}
	}
	if *usePublicIPs == *noUsePublicIPs {
		useSet := isFlagPassed("use_public_ips")
		noUseSet := isFlagPassed("no_use_public_ips")
		// If use_public_ips was explicitly set but no_use_public_ips was not, use that value
		// We take the explicit value of no_use_public_ips if it was set but use_public_ips was not.
		if useSet && !noUseSet {
			*noUsePublicIPs = !*usePublicIPs
		} else if useSet && noUseSet {
			return nil, errors.New("exactly one of usePublicIPs and noUsePublicIPs must be true, please check that only one is true")
		}
	}

	hooks.SerializeHooksToOptions()

	experiments := jobopts.GetExperiments()
	// Ensure that we enable the same set of experiments across all SDKs

View on GitHub (pinned to 12126d8942)

Solutions

Pass a valid JSON object, e.g. --transform_name_mapping='{"oldName":"newName"}'
Validate with `jq .` or a JSON linter before launching
Fix shell quoting so embedded double quotes are preserved
Remember the flag requires --update; if not updating, remove the flag entirely

Example fix

// before
--transform_name_mapping=oldName:newName
// after
--transform_name_mapping='{"oldName":"newName"}'

Defensive patterns

Strategy: validation

Validate before calling

if transformMapping != "" {
    var m map[string]string
    if err := json.Unmarshal([]byte(transformMapping), &m); err != nil {
        return fmt.Errorf("--transform_name_mapping is not valid JSON: %w", err)
    }
}

Try / catch

if err := run(); err != nil {
    if strings.Contains(err.Error(), "--transform_name_mapping") {
        log.Fatalf("fix transform mapping JSON: %v", err)
    }
}

Prevention

Store mappings in a .json file and pass its validated contents
Lint the mapping JSON in CI before pipeline launches
Use single-quoted shell args to preserve double quotes
Only set the flag together with --update

When it happens

Trigger: Running a streaming pipeline update with --update and a --transform_name_mapping value that is not valid JSON (single quotes, unquoted keys, YAML-style indentation, or a non-object JSON value like an array).

Common situations: Hand-editing mapping files and introducing syntax errors, shell quoting stripping double quotes, providing a JSON array instead of an object, or copying mappings from older documentation using a different format.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/072823494c1adf7e. Report an issue: GitHub.