oobabooga/textgen · warning · RuntimeError

unexpected end of input

Error message

unexpected end of input

What it means

Raised by parse_char (modules/grammar/grammar_utils.py:139): the parser is asked for one more character of a quoted literal but the input string is exhausted — typically a literal whose closing quote is missing (or the file was truncated), so the scan runs off the end.

Source

Thrown at modules/grammar/grammar_utils.py:139

            return read_hex(src[2:4]), src[4:]
        elif esc == "u":
            return read_hex(src[2:6]), src[6:]
        elif esc == "U":
            return read_hex(src[2:10]), src[10:]
        elif esc in ('"', "[", "]", "\\", "-"):
            return esc, src[2:]
        elif esc == "r":
            return "\r", src[2:]
        elif esc == "n":
            return "\n", src[2:]
        elif esc == "t":
            return "\t", src[2:]
        elif esc == "\\":
            return "\\", src[2:]
        raise RuntimeError("unknown escape at " + src)
    elif src:
        return src[0], src[1:]
    raise RuntimeError("unexpected end of input")


def parse_sequence(state, src, rule_name, outbuf, is_nested):
    out_start_pos = len(outbuf)

    # sequence size, will be replaced at end when known
    outbuf.append(TO_BE_FILLED_MARKER)

    last_sym_start = len(outbuf)
    remaining_src = src
    while remaining_src:
        if remaining_src[0] == '"':  # literal string
            remaining_src = remaining_src[1:]
            last_sym_start = len(outbuf)
            while remaining_src[0] != '"':
                char, remaining_src = parse_char(remaining_src)

                # each char of a literal is encoded as a "range" of char - char

View on GitHub (pinned to ed888c71f2)

Solutions

1. Find the rule indicated by the surrounding parse position and add the missing closing '"'.
2. Re-download or re-paste the full grammar if it may be truncated.
3. End literals before newlines; GBNF literals cannot span lines.

Example fix

// before
root ::= "hello

// after
root ::= "hello"

import re

def literals_closed(grammar: str) -> bool:
    # strip comments is hard; approximate: count quotes per line must be even
    return all(line.count('"') % 2 == 0
               for line in grammar.splitlines()
               if not line.lstrip().startswith('#'))

try:
    parse_grammar(grammar_text)
except RuntimeError as e:
    if 'unexpected end of input' in str(e):
        raise ValueError('Grammar literal is missing its closing quote') from None
    raise

When it happens

Trigger: A grammar rule like root ::= "unterminated — no closing quote before end of file; or an escape such as \u12 truncated mid-sequence leaving src empty; grammar files with the final line missing its trailing newline/quote after editor truncation.

Common situations: Truncated grammar files (partial download, editor cut-off); copy-paste that dropped the closing '"'; multi-line attempt inside a single-line literal where the quote after the line break is missing.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• expecting name at {src}
• unknown escape at {src}
• expecting ')' at {remaining_src}
• expecting ::= at {remaining_src}
• unknown hex char {c}

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/1d9a2e49c634db0d. Report an issue: GitHub.