antlr/antlr4 · error · ArgumentException

Invalid UTF-16 (high surrogate followed by code point > U+FF

Error message

Invalid UTF-16 (high surrogate followed by code point > U+FFFF

What it means

After reading a high surrogate, UnbufferedCharStream expects a low surrogate as the next UTF-16 code unit. This throw indicates that the value following the high surrogate is greater than a UTF-16 code unit can represent rather than the required low surrogate. It signals a malformed surrogate sequence or an unusual character source.

Source

Thrown at runtime/CSharp/src/UnbufferedCharStream.cs:237

                int c = NextChar();
                if (c > char.MaxValue || c == IntStreamConstants.EOF)
                {
                    Add(c);
                }
                else
                {
                    char ch = unchecked((char)c);
                    if (Char.IsLowSurrogate(ch))
                    {
                        throw new ArgumentException("Invalid UTF-16 (low surrogate with no preceding high surrogate)");
                    }
                    else if (Char.IsHighSurrogate(ch))
                    {
                        int lowSurrogate = NextChar();
                        if (lowSurrogate > char.MaxValue)
                        {
                            throw new ArgumentException("Invalid UTF-16 (high surrogate followed by code point > U+FFFF");
                        }
                        else if (lowSurrogate == IntStreamConstants.EOF)
                        {
                            throw new ArgumentException("Invalid UTF-16 (low surrogate with no preceding high surrogate)");
                        }
                        else
                        {
                            char lowSurrogateChar = unchecked((char)lowSurrogate);
                            if (Char.IsLowSurrogate(lowSurrogateChar))
                            {
                                Add(Char.ConvertToUtf32(ch, lowSurrogateChar));
                            }
                            else
                            {
                                throw new ArgumentException("Invalid UTF-16 (low surrogate with no preceding high surrogate)");
                            }
                        }
                    }

View on GitHub (pinned to 7d5770395b)

Solutions

1. Use a standard StreamReader with a detected or explicit correct encoding.
2. Make custom character sources return UTF-16 code units consistently, never a scalar value after a high surrogate.
3. Repair or reject malformed surrogate sequences before parsing.

Example fix

// before
// Custom reader returns a supplementary code point after a high surrogate.

// after
// Return the pair as two UTF-16 code units: high then low surrogate.

if (source is TextReader reader) {
    using var probe = new StreamReader(new StreamReader(reader).BaseStream, Encoding.UTF8, false, 1024, leaveOpen: true);
    // Prefer a standard reader rather than a custom code-point source.
}

try { stream.Fill(count); }
catch (ArgumentException ex) when (ex.Message.Contains("high surrogate followed by code point")) { /* fix character source */ }

When it happens

Trigger: A high surrogate is followed by a value above char.MaxValue from a custom NextChar()/TextReader source; malformed encoded data produces an invalid unit after a high surrogate; or a custom stream incorrectly returns whole supplementary code points after already returning half of a pair.

Common situations: Custom input adapters that mix UTF-16 code units with Unicode scalar values; broken transcoding pipelines; or corrupt files interpreted as UTF-16.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Invalid UTF-16 (low surrogate with no preceding high surroga
• cannot consume EOF
• release() called with an invalid marker.
• cannot seek to negative index ${index}
• seek to index outside buffer: ${index} not in ${bufferStartI

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/f1648636234565c3. Report an issue: GitHub.