apache/beam · error

unable to json unmarshal --environment_config: %w

Error message

unable to json unmarshal --environment_config: %w

What it means

CreateEnvironment builds the process environment for the pipeline and expects the --environment_config pipeline option to be valid JSON for pipepb.ProcessPayload. protojson.Unmarshal failing means the config string is not valid JSON or does not match the expected message schema. The underlying protojson error is wrapped via %w.

Source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/translate.go:130

		URNWorkerStatus,
		URNMonitoringInfoShortID,
		URNBaseVersionGo,
		URNToString,
		URNDataSampling,
		URNSDKConsumingReceivedData,
	}
	return append(capabilities, knownStandardCoders()...)
}

// CreateEnvironment produces the appropriate payload for the type of environment.
func CreateEnvironment(ctx context.Context, urn string, extractEnvironmentConfig func(context.Context) string) (*pipepb.Environment, error) {
	var serializedPayload []byte
	switch urn {
	case URNEnvProcess:
		config := extractEnvironmentConfig(ctx)
		payload := &pipepb.ProcessPayload{}
		if err := protojson.Unmarshal([]byte(config), payload); err != nil {
			return nil, fmt.Errorf("unable to json unmarshal --environment_config: %w", err)
		}
		serializedPayload = protox.MustEncode(payload)
	case URNEnvExternal:
		config := extractEnvironmentConfig(ctx)
		payload := &pipepb.ExternalPayload{Endpoint: &pipepb.ApiServiceDescriptor{Url: config}}
		serializedPayload = protox.MustEncode(payload)
	case URNEnvDocker:
		fallthrough
	default:
		config := extractEnvironmentConfig(ctx)
		payload := &pipepb.DockerPayload{ContainerImage: config}
		serializedPayload = protox.MustEncode(payload)
	}
	return &pipepb.Environment{
		Urn:          urn,
		Payload:      serializedPayload,
		Capabilities: goCapabilities(),
		Dependencies: []*pipepb.ArtifactInformation{

View on GitHub (pinned to 12126d8942)

Solutions

1. Validate the --environment_config value is well-formed JSON matching pipepb.ProcessPayload fields (e.g. {"os":"linux","arch":"amd64"}).
2. Use double quotes and correct casing for field names per protojson rules.
3. Omit the option if defaults are acceptable — extractEnvironmentConfig presumably yields a default when unset.
4. Check SDK version compatibility between the runner emitting the config and the Go SDK parsing it.

Example fix

// before
--environment_config="{'os':'linux'}"
// after
--environment_config='{"os":"linux","arch":"amd64"}'

var probe map[string]any
if err := json.Unmarshal([]byte(envConfig), &probe); err != nil {
	return fmt.Errorf("--environment_config is not valid JSON: %w", err)
}

if err != nil {
	return nil, fmt.Errorf("bad --environment_config (%q): %w", config, err)
}

When it happens

Trigger: Running with URNEnvProcess where the --environment_config option (fetched by extractEnvironmentConfig from the context/pipeline options) is malformed JSON, empty, or uses fields incompatible with ProcessPayload.

Common situations: Hand-written runner invocations passing --environment_config=... with typos, single quotes instead of JSON double quotes, missing config entirely, or SDK/runner version drift where the config schema changed.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• invalid coder
• Could not unmarshal SourceConfig: %v
• unparsable resource hint: %q
• unknown resource hint: %v
• unmarshalling partitionFn data

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2f1b9feb8e000957. Report an issue: GitHub.