apache/beam · error · IllegalArgumentException

illegal codepoint

Error message

illegal codepoint

What it means

unescapeOctal converts an octal digit sequence to a code point and rejects results that are not valid Unicode code points. An octal escape like \777 evaluates to 511 which is valid, but sequences producing values above 0x10FFFF (the max code point) or in the surrogate range fail Character.isValidCodePoint and throw IllegalArgumentException("illegal codepoint").

Solutions

  1. Fix the octal escape sequence so it decodes to a valid Unicode code point (0x0–0x10FFFF, excluding surrogates).
  2. Prefer \uXXXX hex escapes for characters instead of octal to avoid ambiguity.
  3. Validate escaped field names at ingestion time before they reach Firestore queries.
  4. Re-encode the field name with the library's own escaping routine instead of manual escaping.

Example fix

// before
String field = "bad\4177777name"; // octal value exceeds max code point
// after
String field = "bad\uFFFDname"; // valid hex-encoded replacement char
Defensive patterns

Strategy: validation

Validate before calling

// Java: validate octal escapes decode to valid code points before use
static boolean octalEscapesValid(String name) {
  java.util.regex.Matcher m =
      java.util.regex.Pattern.compile("\\\\([0-7]{1,6})").matcher(name);
  while (m.find()) {
    int v = Integer.parseInt(m.group(1), 8);
    if (!Character.isValidCodePoint(v)) return false;
  }
  return true;
}

Try / catch

try {
  String decoded = QueryUtils.unescaped(fieldName);
} catch (IllegalArgumentException e) {
  LOG.warn("Invalid codepoint escape in field name: %s", fieldName, e);
}

Prevention

When it happens

Trigger: A field name with an octal escape sequence whose accumulated value is not a valid Unicode code point (e.g. very long octal runs like \4177777 that exceed 0x10FFFF) reached during unescapeFieldName.

Common situations: Corrupted or hand-edited escaped field names; escaping tools that emit octal for arbitrary byte values including surrogates; names round-tripped through encoders with different escape semantics.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/529ae1aee1243d55. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/QueryUtils.java:361

              }
              buf.appendCodePoint(unescapeHex(fieldName.substring(i, i + 8)));
              i += 8;
              break;
            default:
              throw new IllegalArgumentException("illegal escape");
          }
        }
      }
      return buf.toString();
    }

    private static int unescapeOctal(String str) {
      int ch = 0;
      for (int i = 0; i < str.length(); i++) {
        ch = 8 * ch + octalValue(str.charAt(i));
      }
      if (!Character.isValidCodePoint(ch)) {
        throw new IllegalArgumentException("illegal codepoint");
      }
      return ch;
    }

    private static int unescapeHex(String str) {
      int ch = 0;
      for (int i = 0; i < str.length(); i++) {
        ch = 16 * ch + hexValue(str.charAt(i));
      }
      if (!Character.isValidCodePoint(ch)) {
        throw new IllegalArgumentException("illegal codepoint");
      }
      return ch;
    }

    private static int octalValue(char d) {
      if (d >= '0' && d <= '7') {
        return d - '0';

View on GitHub (pinned to 12126d8942)