{"record":{"id":"1daa38031773049f","repo":"antlr/antlr4","slug":"invalid-utf-16-dangling-high-surrogate","errorCode":null,"errorMessage":"Invalid UTF-16 (dangling high surrogate","messagePattern":"Invalid UTF-16 \\(dangling high surrogate","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"runtime/Java/src/org/antlr/v4/runtime/UnbufferedCharStream.java","lineNumber":188,"sourceCode":"\t\t\t\t\tchar ch = (char) c;\n\t\t\t\t\tif (Character.isLowSurrogate(ch)) {\n\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (low surrogate with no preceding high surrogate)\");\n\t\t\t\t\t}\n\t\t\t\t\telse if (Character.isHighSurrogate(ch)) {\n\t\t\t\t\t\tint lowSurrogate = nextChar();\n\t\t\t\t\t\tif (lowSurrogate > Character.MAX_VALUE) {\n\t\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (high surrogate followed by code point > U+FFFF\");\n\t\t\t\t\t\t}\n\t\t\t\t\t\telse if (lowSurrogate == IntStream.EOF) {\n\t\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (dangling high surrogate at end of file)\");\n\t\t\t\t\t\t}\n\t\t\t\t\t\telse {\n\t\t\t\t\t\t\tchar lowSurrogateChar = (char) lowSurrogate;\n\t\t\t\t\t\t\tif (Character.isLowSurrogate(lowSurrogateChar)) {\n\t\t\t\t\t\t\t\tadd(Character.toCodePoint(ch, lowSurrogateChar));\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\telse {\n\t\t\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (dangling high surrogate\");\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\telse {\n\t\t\t\t\t\tadd(c);\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tcatch (IOException ioe) {\n\t\t\t\tthrow new RuntimeException(ioe);\n\t\t\t}\n\t\t}\n\n\t\treturn n;\n\t}\n\n\t/**\n\t * Override to provide different source of characters than","sourceCodeStart":170,"sourceCodeEnd":206,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/UnbufferedCharStream.java#L170-L206","documentation":"UnbufferedCharStream read a high surrogate but the following character was not a low surrogate (and not EOF), so the pair is dangling. The stream throws because it cannot form a code point and will not store unpaired surrogates. As with the related errors, the usual root cause is an encoding mismatch between the Reader and the actual bytes.","triggerScenarios":"UTF-8 or single-byte content opened through a UTF-16 Reader so random adjacent characters land after a 0xD800..0xDBFF value; content containing literal escaped surrogates; transcode pipelines that split pairs.","commonSituations":"Default-charset Readers on JVMs where the platform encoding differs from the file's; concatenating strings that each hold half of a surrogate pair.","solutions":["Open the Reader with the explicit correct charset (StandardCharsets.UTF_8 for typical text files)","Validate/sanitize the input for unpaired surrogates before lexing if the source is untrusted","Re-encode the file to UTF-8 once, then always read it as UTF-8"],"exampleFix":"// before\nnew UnbufferedCharStream(new FileReader(f)); // platform-default charset\n\n// after\nnew UnbufferedCharStream(Files.newBufferedReader(f.toPath(), StandardCharsets.UTF_8));","handlingStrategy":"try-catch","validationCode":"static boolean hasUnpairedSurrogates(String s) {\n    for (int i = 0; i < s.length(); i++) {\n        char c = s.charAt(i);\n        if (Character.isHighSurrogate(c)\n            && (i + 1 >= s.length() || !Character.isLowSurrogate(s.charAt(i + 1)))) return true;\n        if (Character.isLowSurrogate(c)\n            && (i == 0 || !Character.isHighSurrogate(s.charAt(i - 1)))) return true;\n    }\n    return false;\n}","typeGuard":null,"tryCatchPattern":"try { parse(stream); }\ncatch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"UTF-16\"))\n        return Result.badEncoding();\n    throw e;\n}","preventionTips":["Read files via Files.newBufferedReader with an explicit charset","Never build Readers with the platform default charset","Sanitize untrusted text for unpaired surrogates before lexing"],"tags":["antlr","char-stream","utf-16","encoding","surrogate"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}