apache/beam · error · IllegalArgumentException

not in legal encoded format; substring

Error message

not in legal encoded format; substring [{i}..{i+2}] not in format "%xx"

What it means

StringUtils.jsonStringToByteArray decodes strings where bytes are escaped as '%xx'. When it encounters a '%' that is not followed by two hex digits, it throws this IllegalArgumentException. The input string is not in the escaped encoded format the function expects.

Solutions

1. Escape '%' characters correctly before calling, using the matching byteArrayToJsonString method, so every '%' is followed by two hex digits.
2. Validate that each '%' in the input is followed by exactly two hex characters before invoking jsonStringToByteArray.
3. If the input is raw user data (e.g. a filename with '%'), sanitize or percent-encode it first.
4. Ensure you are not double-processing a string that was already decoded (a lone '%' remains).

Example fix

// before
byte[] bytes = StringUtils.jsonStringToByteArray("100% done"); // '%' not followed by hex
// after
String escaped = StringUtils.byteArrayToJsonString("100% done".getBytes(StandardCharsets.UTF_8));
byte[] bytes = StringUtils.jsonStringToByteArray(escaped); // round-trips correctly

static boolean isLegalEscapedFormat(String s) {
  for (int i = 0; i < s.length(); i++) {
    if (s.charAt(i) == '%' && (i + 2 >= s.length()
        || !isHex(s.charAt(i + 1)) || !isHex(s.charAt(i + 2)))) {
      return false;
    }
  }
  return true;
}

try {
  bytes = StringUtils.jsonStringToByteArray(s);
} catch (IllegalArgumentException e) {
  // re-escape input and retry
}

When it happens

Trigger: Calling jsonStringToByteArray on a string containing a '%' at index i where substring(i+1, i+3) is shorter than two characters or is not valid hex (e.g. "100%", "%G1", "%a", or a raw unescaped percent in a filename).

Common situations: Passing plain file names or user strings containing literal '%' through Beam's JSON-string escaping path (e.g. WindowedFilenamePolicy or dynamic destinations) instead of pre-escaped values; hand-built escaped strings with malformed escapes.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• encoding , expected
• err
• failed encoding key for
• Failed to encode key
• Failed to encode values for multimap user state id

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/00ada86a286ccc4f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/StringUtils.java:71

    return sb.toString();
  }

  /**
   * Converts the given string, encoded using {@link #byteArrayToJsonString}, into a byte array.
   *
   * @throws IllegalArgumentException if the argument string is not legal
   */
  public static byte[] jsonStringToByteArray(String string) {
    List<Byte> bytes = new ArrayList<>();
    for (int i = 0; i < string.length(); ) {
      char c = string.charAt(i);
      Byte b;
      if (c == '%') {
        // Escaped.  Expect '%xx' format.
        try {
          b = (byte) Integer.parseInt(string.substring(i + 1, i + 3), 16);
        } catch (IndexOutOfBoundsException | NumberFormatException exn) {
          throw new IllegalArgumentException(
              "not in legal encoded format; "
                  + "substring ["
                  + i
                  + ".."
                  + (i + 2)
                  + "] not in format \"%xx\"",
              exn);
        }
        i += 3;
      } else {
        // Send through unchanged.
        b = (byte) c;
        i++;
      }
      bytes.add(b);
    }
    byte[] byteArray = new byte[bytes.size()];
    int i = 0;

View on GitHub (pinned to 12126d8942)