{"record":{"id":"50d786f16cd92c78","repo":"antlr/antlr4","slug":"invalid-utf-16-dangling-high-surrogate-at-end-of","errorCode":null,"errorMessage":"Invalid UTF-16 (dangling high surrogate at end of file)","messagePattern":"Invalid UTF-16 \\(dangling high surrogate at end of file\\)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"runtime/Java/src/org/antlr/v4/runtime/UnbufferedCharStream.java","lineNumber":180,"sourceCode":"\t\t\t}\n\n\t\t\ttry {\n\t\t\t\tint c = nextChar();\n\t\t\t\tif (c > Character.MAX_VALUE || c == IntStream.EOF) {\n\t\t\t\t\tadd(c);\n\t\t\t\t}\n\t\t\t\telse {\n\t\t\t\t\tchar ch = (char) c;\n\t\t\t\t\tif (Character.isLowSurrogate(ch)) {\n\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (low surrogate with no preceding high surrogate)\");\n\t\t\t\t\t}\n\t\t\t\t\telse if (Character.isHighSurrogate(ch)) {\n\t\t\t\t\t\tint lowSurrogate = nextChar();\n\t\t\t\t\t\tif (lowSurrogate > Character.MAX_VALUE) {\n\t\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (high surrogate followed by code point > U+FFFF\");\n\t\t\t\t\t\t}\n\t\t\t\t\t\telse if (lowSurrogate == IntStream.EOF) {\n\t\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (dangling high surrogate at end of file)\");\n\t\t\t\t\t\t}\n\t\t\t\t\t\telse {\n\t\t\t\t\t\t\tchar lowSurrogateChar = (char) lowSurrogate;\n\t\t\t\t\t\t\tif (Character.isLowSurrogate(lowSurrogateChar)) {\n\t\t\t\t\t\t\t\tadd(Character.toCodePoint(ch, lowSurrogateChar));\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\telse {\n\t\t\t\t\t\t\t\tthrow new RuntimeException(\"Invalid UTF-16 (dangling high surrogate\");\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\telse {\n\t\t\t\t\t\tadd(c);\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tcatch (IOException ioe) {\n\t\t\t\tthrow new RuntimeException(ioe);","sourceCodeStart":162,"sourceCodeEnd":198,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/UnbufferedCharStream.java#L162-L198","documentation":"UnbufferedCharStream read a high surrogate as the final character of the input: the next read returned EOF, so the pair is incomplete. This is malformed UTF-16 (a dangling high surrogate at end of file) and the stream throws rather than buffer an unpaired surrogate.","triggerScenarios":"Input whose last char is a lone high surrogate: truncated multi-byte character at EOF, a file cut mid-write, or single-byte encoding misread as UTF-16 so the final odd byte becomes half a pair.","commonSituations":"Truncated downloads or log files ending mid-character; pre-processing steps that slice strings at arbitrary offsets and cut a surrogate pair in half.","solutions":["Repair or re-fetch the truncated input file","If slicing text before lexing, cut on code-point boundaries (e.g. offsetByCodePoints), never on raw char indexes","Confirm the Reader charset matches the file (see error 47) so pairs are not created spuriously"],"exampleFix":"// before\nString chunk = input.substring(0, 1000); // may cut a surrogate pair\nCharStream cs = new UnbufferedCharStream(new StringReader(chunk));\n\n// after\nint end = input.offsetByCodePoints(0, 1000); // code-point boundary\nCharStream cs = new UnbufferedCharStream(new StringReader(input.substring(0, end)));","handlingStrategy":"try-catch","validationCode":"// check the tail of the string for a dangling high surrogate before lexing\nString s = readAll(input);\nif (!s.isEmpty() && Character.isHighSurrogate(s.charAt(s.length() - 1))) {\n    throw new IllegalArgumentException(\"input ends with a truncated surrogate pair\");\n}","typeGuard":null,"tryCatchPattern":"try { parse(stream); }\ncatch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"dangling high surrogate\"))\n        return Result.truncatedInput(); // treat as incomplete data, request re-upload\n    throw e;\n}","preventionTips":["Slice strings on code-point boundaries (offsetByCodePoints)","Detect truncated files (size checks, checksums) before parsing","Keep surrogate pairs together in any pre-lexer text processing"],"tags":["antlr","char-stream","utf-16","truncation","surrogate"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}