apache/beam · error · IllegalArgumentException

Failed to parse hadoop_config string as JSON

Error message

Failed to parse hadoop_config string as JSON

What it means

DeltaTable.parseHadoopConfig expects the 'hadoop_config' property value to be a JSON object string mapping config keys to string values. If Jackson's ObjectMapper fails to deserialize the text into Map<String, String>, the exception is wrapped and rethrown as IllegalArgumentException with this message. It means your hadoop_config value is not valid JSON of the expected shape.

Source

Thrown at sdks/java/extensions/sql/delta/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/delta/DeltaTable.java:135

  private static void parseHadoopConfig(JsonNode val, Map<String, String> targetMap) {
    if (val.isObject()) {
      Map<String, String> map =
          TableUtils.getObjectMapper()
              .convertValue(val, new TypeReference<Map<String, String>>() {});
      if (map != null) {
        targetMap.putAll(map);
      }
    } else if (val.isTextual()) {
      try {
        Map<String, String> map =
            TableUtils.getObjectMapper()
                .readValue(val.asText(), new TypeReference<Map<String, String>>() {});
        if (map != null) {
          targetMap.putAll(map);
        }
      } catch (Exception e) {
        throw new IllegalArgumentException("Failed to parse hadoop_config string as JSON", e);
      }
    }
  }

  @Override
  public PCollection<Row> buildIOReader(PBegin begin) {
    return begin
        .apply(Managed.read(Managed.DELTA_LAKE).withConfig(getBaseConfig()))
        .getSinglePCollection();
  }

  @Override
  public PCollection<Row> buildIOReader(
      PBegin begin, BeamSqlTableFilter filters, List<String> fieldNames) {
    // TODO: Support predicate pushdown and column pruning when supported by DeltaIO
    // / Managed Delta
    // Lake source.
    String error = "%s does not support predicate/project push-down, yet non-empty %s is passed.";

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure hadoop_config is a valid JSON object with only string values, e.g. '{"fs.defaultFS":"hdfs://nn:8020"}'.
2. Validate the JSON with a parser/linter before embedding it in the table properties.
3. Escape quotes correctly when embedding JSON inside DDL strings.

Example fix

// before
TBLPROPERTIES { 'hadoop_config': 'fs.defaultFS=hdfs://nn:8020' }
// after
TBLPROPERTIES { 'hadoop_config': '{"fs.defaultFS":"hdfs://nn:8020"}' }

// validate hadoop_config before passing it
new ObjectMapper().readValue(hadoopConfigJson, new TypeReference<Map<String, String>>() {}); // throws if invalid

boolean isValidHadoopConfig(String s) {
  try {
    Map<String, String> m = new ObjectMapper().readValue(s, new TypeReference<Map<String, String>>() {});
    return m != null;
  } catch (Exception e) { return false; }
}

try {
  table = new DeltaTable(tableId, schema, properties);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("hadoop_config")) {
    // validate and fix the JSON, then retry
  }
}

When it happens

Trigger: Passing a 'hadoop_config' property whose value is malformed JSON, JSON with non-string values (e.g. numbers/booleans), or a non-JSON plain string like 'fs.defaultFS=hdfs://...'.

Common situations: Hand-writing the hadoop_config inline in DDL with quoting/escaping mistakes; using properties-file syntax instead of JSON; nested objects or numeric values that don't fit Map<String, String>.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Beam write property '%s' is not supported. Writing to Delta
• Unknown Beam read property: {key}
• Unknown property '%s'
• Cannot set both version and timestamp.
• %s does not support predicate/project push-down, yet non-emp

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9537f4241bcbe09e. Report an issue: GitHub.