{"record":{"id":"f1648636234565c3","repo":"antlr/antlr4","slug":"invalid-utf-16-high-surrogate-followed-by-code-po-f16486","errorCode":null,"errorMessage":"Invalid UTF-16 (high surrogate followed by code point > U+FFFF","messagePattern":"Invalid UTF-16 \\(high surrogate followed by code point > U\\+FFFF","errorType":"exception","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/UnbufferedCharStream.cs","lineNumber":237,"sourceCode":"\n                int c = NextChar();\n                if (c > char.MaxValue || c == IntStreamConstants.EOF)\n                {\n                    Add(c);\n                }\n                else\n                {\n                    char ch = unchecked((char)c);\n                    if (Char.IsLowSurrogate(ch))\n                    {\n                        throw new ArgumentException(\"Invalid UTF-16 (low surrogate with no preceding high surrogate)\");\n                    }\n                    else if (Char.IsHighSurrogate(ch))\n                    {\n                        int lowSurrogate = NextChar();\n                        if (lowSurrogate > char.MaxValue)\n                        {\n                            throw new ArgumentException(\"Invalid UTF-16 (high surrogate followed by code point > U+FFFF\");\n                        }\n                        else if (lowSurrogate == IntStreamConstants.EOF)\n                        {\n                            throw new ArgumentException(\"Invalid UTF-16 (low surrogate with no preceding high surrogate)\");\n                        }\n                        else\n                        {\n                            char lowSurrogateChar = unchecked((char)lowSurrogate);\n                            if (Char.IsLowSurrogate(lowSurrogateChar))\n                            {\n                                Add(Char.ConvertToUtf32(ch, lowSurrogateChar));\n                            }\n                            else\n                            {\n                                throw new ArgumentException(\"Invalid UTF-16 (low surrogate with no preceding high surrogate)\");\n                            }\n                        }\n                    }","sourceCodeStart":219,"sourceCodeEnd":255,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/UnbufferedCharStream.cs#L219-L255","documentation":"After reading a high surrogate, UnbufferedCharStream expects a low surrogate as the next UTF-16 code unit. This throw indicates that the value following the high surrogate is greater than a UTF-16 code unit can represent rather than the required low surrogate. It signals a malformed surrogate sequence or an unusual character source.","triggerScenarios":"A high surrogate is followed by a value above char.MaxValue from a custom NextChar()/TextReader source; malformed encoded data produces an invalid unit after a high surrogate; or a custom stream incorrectly returns whole supplementary code points after already returning half of a pair.","commonSituations":"Custom input adapters that mix UTF-16 code units with Unicode scalar values; broken transcoding pipelines; or corrupt files interpreted as UTF-16.","solutions":["Use a standard StreamReader with a detected or explicit correct encoding.","Make custom character sources return UTF-16 code units consistently, never a scalar value after a high surrogate.","Repair or reject malformed surrogate sequences before parsing."],"exampleFix":"// before\n// Custom reader returns a supplementary code point after a high surrogate.\n\n// after\n// Return the pair as two UTF-16 code units: high then low surrogate.","handlingStrategy":"validation","validationCode":"if (source is TextReader reader) {\n    using var probe = new StreamReader(new StreamReader(reader).BaseStream, Encoding.UTF8, false, 1024, leaveOpen: true);\n    // Prefer a standard reader rather than a custom code-point source.\n}","typeGuard":null,"tryCatchPattern":"try { stream.Fill(count); }\ncatch (ArgumentException ex) when (ex.Message.Contains(\"high surrogate followed by code point\")) { /* fix character source */ }","preventionTips":["Custom character sources must emit UTF-16 code units only.","Do not emit whole supplementary code points after a high surrogate.","Prefer StreamReader for file/network input."],"tags":["antlr","csharp","unicode","utf-16","streaming"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}