{"record":{"id":"529ae1aee1243d55","repo":"apache/beam","slug":"illegal-codepoint","errorCode":null,"errorMessage":"illegal codepoint","messagePattern":"illegal codepoint","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/QueryUtils.java","lineNumber":361,"sourceCode":"              }\n              buf.appendCodePoint(unescapeHex(fieldName.substring(i, i + 8)));\n              i += 8;\n              break;\n            default:\n              throw new IllegalArgumentException(\"illegal escape\");\n          }\n        }\n      }\n      return buf.toString();\n    }\n\n    private static int unescapeOctal(String str) {\n      int ch = 0;\n      for (int i = 0; i < str.length(); i++) {\n        ch = 8 * ch + octalValue(str.charAt(i));\n      }\n      if (!Character.isValidCodePoint(ch)) {\n        throw new IllegalArgumentException(\"illegal codepoint\");\n      }\n      return ch;\n    }\n\n    private static int unescapeHex(String str) {\n      int ch = 0;\n      for (int i = 0; i < str.length(); i++) {\n        ch = 16 * ch + hexValue(str.charAt(i));\n      }\n      if (!Character.isValidCodePoint(ch)) {\n        throw new IllegalArgumentException(\"illegal codepoint\");\n      }\n      return ch;\n    }\n\n    private static int octalValue(char d) {\n      if (d >= '0' && d <= '7') {\n        return d - '0';","sourceCodeStart":343,"sourceCodeEnd":379,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/QueryUtils.java#L343-L379","documentation":"unescapeOctal converts an octal digit sequence to a code point and rejects results that are not valid Unicode code points. An octal escape like \\777 evaluates to 511 which is valid, but sequences producing values above 0x10FFFF (the max code point) or in the surrogate range fail Character.isValidCodePoint and throw IllegalArgumentException(\"illegal codepoint\").","triggerScenarios":"A field name with an octal escape sequence whose accumulated value is not a valid Unicode code point (e.g. very long octal runs like \\4177777 that exceed 0x10FFFF) reached during unescapeFieldName.","commonSituations":"Corrupted or hand-edited escaped field names; escaping tools that emit octal for arbitrary byte values including surrogates; names round-tripped through encoders with different escape semantics.","solutions":["Fix the octal escape sequence so it decodes to a valid Unicode code point (0x0–0x10FFFF, excluding surrogates).","Prefer \\uXXXX hex escapes for characters instead of octal to avoid ambiguity.","Validate escaped field names at ingestion time before they reach Firestore queries.","Re-encode the field name with the library's own escaping routine instead of manual escaping."],"exampleFix":"// before\nString field = \"bad\\4177777name\"; // octal value exceeds max code point\n// after\nString field = \"bad\\uFFFDname\"; // valid hex-encoded replacement char","handlingStrategy":"validation","validationCode":"// Java: validate octal escapes decode to valid code points before use\nstatic boolean octalEscapesValid(String name) {\n  java.util.regex.Matcher m =\n      java.util.regex.Pattern.compile(\"\\\\\\\\([0-7]{1,6})\").matcher(name);\n  while (m.find()) {\n    int v = Integer.parseInt(m.group(1), 8);\n    if (!Character.isValidCodePoint(v)) return false;\n  }\n  return true;\n}","typeGuard":null,"tryCatchPattern":"try {\n  String decoded = QueryUtils.unescaped(fieldName);\n} catch (IllegalArgumentException e) {\n  LOG.warn(\"Invalid codepoint escape in field name: %s\", fieldName, e);\n}","preventionTips":["Avoid octal escapes; use \\uXXXX hex escapes which bound the value range.","Round-trip test every escaped field name through encode/decode.","Do not hex/octal-encode raw bytes as if they were code points."],"tags":["firestore","field-name","escaping","unicode","java"],"backgroundTag":"invalid-argument-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}