apache/beam · error · RuntimeException

Unable to parse %s

Error message

Unable to parse %s

What it means

updateSerializedOptions parses a serialized PipelineOptions JSON, expecting an object with an 'options' member. This error wraps any IOException from Jackson while reading that JSON (or the ObjectNode conversion), e.g. malformed JSON or a non-object document. It lets callers know the runtime value-provider template string could not be parsed.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/options/ValueProviders.java:46

  private ValueProviders() {}

  /**
   * Given {@code serializedOptions} as a JSON-serialized {@link PipelineOptions}, updates the
   * values according to the provided values in {@code runtimeValues}.
   *
   * @deprecated Use {@link org.apache.beam.sdk.testing.TestPipeline#newProvider} for testing {@link
   *     ValueProvider} code.
   */
  @Deprecated
  public static String updateSerializedOptions(
      String serializedOptions, Map<String, String> runtimeValues) {
    ObjectNode root, options;
    try {
      root = PipelineOptionsFactory.MAPPER.readValue(serializedOptions, ObjectNode.class);
      options = (ObjectNode) root.get("options");
      checkStateNotNull(options, "Unable to locate 'options' in %s", serializedOptions);
    } catch (IOException e) {
      throw new RuntimeException(String.format("Unable to parse %s", serializedOptions), e);
    }

    for (Map.Entry<String, String> entry : runtimeValues.entrySet()) {
      options.put(entry.getKey(), entry.getValue());
    }
    try {
      return PipelineOptionsFactory.MAPPER.writeValueAsString(root);
    } catch (IOException e) {
      throw new RuntimeException("Unable to parse re-serialize options", e);
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Validate the serializedOptions string is well-formed JSON and has a top-level 'options' object before calling.
2. Regenerate the serialization with PipelineOptionsFactory.as(...).toString() / the SDK's serializer instead of hand-crafting it.
3. Inspect the wrapped IOException cause for the exact parse failure position and fix that offset.
4. If the string comes from a template parameter, ensure the template engine did not HTML-escape or mangle quotes.

Example fix

// before
String opts = "pipeline-args"; // not JSON
ValueProviders.updateSerializedOptions(opts, values);

// after
String opts = PipelineOptionsFactory.fromArgs(args).as(MyOptions.class).toString();
ValueProviders.updateSerializedOptions(opts, values);

new ObjectMapper().readTree(serializedOptions).withArrayOrObject("options"); // fail fast if not an object with 'options'

static boolean isOptionsTemplate(String s) {
  try { return new ObjectMapper().readTree(s).has("options"); } catch (Exception e) { return false; }
}

try { ValueProviders.updateSerializedOptions(opts, values); }
catch (RuntimeException e) {
  // cause is IOException from Jackson
  throw new IllegalArgumentException("Bad serialized options template", e.getCause());
}

When it happens

Trigger: Calling ValueProviders.updateSerializedOptions(serializedOptions, runtimeValues) where serializedOptions is not valid JSON or cannot be read as an ObjectNode — e.g. a corrupted or hand-edited template serialization, or a string that is not the output of PipelineOptionsFactory serialization.

Common situations: Templated Dataflow pipelines where the --options serialization was truncated or altered by shell escaping; passing a plain string/URL instead of the serialized options JSON; version mismatch producing JSON with unexpected top-level structure.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Found incorrectly annotated property methods, if a method is
• Unable to parse JSON value
• Unable to parse re-serialize options
• Unable to parse Row
• Failed to read PipelineOptions from JSON

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0b1c193965b3e495. Report an issue: GitHub.