apache/beam · error · RuntimeException

Failed to read PipelineOptions from JSON

Error message

Failed to read PipelineOptions from JSON

What it means

fromJson parses an options JSON string: modern format is a JSON object with 'options' key (URN-keyed) or a plain Protobuf Struct JSON. After determining the format and delegating to fromProto, any IOException from Jackson or JsonFormat parsing is wrapped in a RuntimeException with this message.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/PipelineOptionsTranslation.java:126

    }
  }

  /** Converts the provided Json{@link String} into {@link PipelineOptions}. */
  public static PipelineOptions fromJson(String optionsJson) {
    try {
      Map<String, Object> probingOptionsMap =
          MAPPER.readValue(optionsJson, new TypeReference<Map<String, Object>>() {});
      if (probingOptionsMap.containsKey("options")) {
        // Legacy options.
        return MAPPER.readValue(optionsJson, PipelineOptions.class);
      } else {
        // Fn Options with namespace and version.
        Struct.Builder builder = Struct.newBuilder();
        JsonFormat.parser().merge(optionsJson, builder);
        return fromProto(builder.build());
      }
    } catch (IOException e) {
      throw new RuntimeException("Failed to read PipelineOptions from JSON", e);
    }
  }

  /** Converts the provided {@link PipelineOptions} into Json{@link String}. */
  public static String toJson(PipelineOptions options) {
    try {
      return JsonFormat.printer().print(toProto(options));
    } catch (InvalidProtocolBufferException e) {
      throw new RuntimeException("Failed to convert PipelineOptions to JSON", e);
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Validate the options JSON with a JSON parser/linter before submitting
2. Ensure the JSON matches the expected shape: {"options": {"beam:option:...": value}} or a valid Struct object
3. Strip any non-JSON bytes (BOM, comments) and re-run fromJson

Example fix

// before
String json = "{options: {beam:option:appName}}}"; // malformed
// after
String json = "{\"options\": {\"beam:option:app_name:v1\": \"myapp\"}}";

new com.fasterxml.jackson.databind.ObjectMapper().readTree(optionsJson); // throws JsonProcessingException if malformed

boolean isJsonObject(String s) { try { return new ObjectMapper().readTree(s).isObject(); } catch (Exception e) { return false; } }

try { PipelineOptions o = PipelineOptionsTranslation.fromJson(json); } catch (RuntimeException e) { log.error("bad options JSON: {}", json, e.getCause()); throw new IllegalArgumentException("Invalid pipeline options JSON", e); }

When it happens

Trigger: Calling PipelineOptionsTranslation.fromJson with malformed JSON, or JSON whose shape matches neither the 'options' keyed format nor a valid Struct, causing JsonFormat.parser().merge or the Jackson read to throw IOException.

Common situations: Passing hand-written or truncated options JSON to job submission; a corrupted --pipelineOptions CLI value; copy-pasted JSON with BOM or syntax errors.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Related errors

• Found incorrectly annotated property methods, if a method is
• Property [%s] is marked with contradictory annotations. Foun
• All inherited interfaces of [%s] should inherit from the Pip
• Method [%s] has multiple definitions %s with different retur
• Interface [%s] has Methods with multiple definitions with di

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fdece3d1e4ed87c2. Report an issue: GitHub.