antlr/antlr4 · error · ArgumentException

Invalid UTF-16 (low surrogate with no preceding high surroga

Error message

Invalid UTF-16 (low surrogate with no preceding high surrogate)

What it means

While filling its buffer, UnbufferedCharStream decodes UTF-16 code units and validates surrogate pairs. This throw occurs when the next code unit is a low surrogate (U+DC00..U+DFFF) but no preceding high surrogate was seen, so it cannot form a valid Unicode scalar value.

Source

Thrown at runtime/CSharp/src/UnbufferedCharStream.cs:230

        {
            for (int i = 0; i < n; i++)
            {
                if (this.n > 0 && data[this.n - 1] == IntStreamConstants.EOF)
                {
                    return i;
                }

                int c = NextChar();
                if (c > char.MaxValue || c == IntStreamConstants.EOF)
                {
                    Add(c);
                }
                else
                {
                    char ch = unchecked((char)c);
                    if (Char.IsLowSurrogate(ch))
                    {
                        throw new ArgumentException("Invalid UTF-16 (low surrogate with no preceding high surrogate)");
                    }
                    else if (Char.IsHighSurrogate(ch))
                    {
                        int lowSurrogate = NextChar();
                        if (lowSurrogate > char.MaxValue)
                        {
                            throw new ArgumentException("Invalid UTF-16 (high surrogate followed by code point > U+FFFF");
                        }
                        else if (lowSurrogate == IntStreamConstants.EOF)
                        {
                            throw new ArgumentException("Invalid UTF-16 (low surrogate with no preceding high surrogate)");
                        }
                        else
                        {
                            char lowSurrogateChar = unchecked((char)lowSurrogate);
                            if (Char.IsLowSurrogate(lowSurrogateChar))
                            {
                                Add(Char.ConvertToUtf32(ch, lowSurrogateChar));

View on GitHub (pinned to 7d5770395b)

Solutions

Fix the input so every low surrogate is preceded by a matching high surrogate.
Validate or re-encode the source as well-formed UTF-8/UTF-16 before creating the stream.
Avoid substring operations that can split surrogate pairs.
Audit custom TextReader/ICharStream adapters for malformed code-unit sequencing.

Example fix

// before
var bad = "\uDC00";
var stream = new UnbufferedCharStream(new StringReader(bad));

// after
var good = "\U0001F600"; // complete surrogate pair
var stream = new UnbufferedCharStream(new StringReader(good));

Defensive patterns

Strategy: validation

Validate before calling

static bool HasValidUtf16Pairs(string s) {
    for (int i = 0; i < s.Length; i++) {
        if (char.IsLowSurrogate(s[i]) && (i == 0 || !char.IsHighSurrogate(s[i-1]))) return false;
        if (char.IsHighSurrogate(s[i]) && (i + 1 >= s.Length || !char.IsLowSurrogate(s[i+1]))) return false;
    }
    return true;
}

Try / catch

try { var stream = new UnbufferedCharStream(new StringReader(text)); stream.Fill(1); }
catch (ArgumentException ex) when (ex.Message.Contains("Invalid UTF-16")) { /* reject malformed input */ }

Prevention

Use UTF8/Unicode decoders that throw on invalid sequences.
Never split strings between surrogate halves.
Validate externally sourced text before parsing.

When it happens

Trigger: The underlying TextReader returns a string such as "\uDC00" at the current position; a surrogate pair is split and only its low half reaches the stream; or malformed UTF-16 is decoded without error detection before ANTLR reads it.

Common situations: Corrupt source files; incorrect encoding conversion; slicing strings in the middle of surrogate pairs; network or database data containing unpaired surrogates; or custom TextReader implementations.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/6bb33a063de58ae8. Report an issue: GitHub.