oobabooga/textgen · warning · RuntimeError

unexpected end of input

Error message

unexpected end of input

What it means

Raised by parse_char (modules/grammar/grammar_utils.py:139): the parser is asked for one more character of a quoted literal but the input string is exhausted — typically a literal whose closing quote is missing (or the file was truncated), so the scan runs off the end.

Solutions

1. Find the rule indicated by the surrounding parse position and add the missing closing '"'.
2. Re-download or re-paste the full grammar if it may be truncated.
3. End literals before newlines; GBNF literals cannot span lines.

Example fix

// before
root ::= "hello

// after
root ::= "hello"

import re

def literals_closed(grammar: str) -> bool:
    # strip comments is hard; approximate: count quotes per line must be even
    return all(line.count('"') % 2 == 0
               for line in grammar.splitlines()
               if not line.lstrip().startswith('#'))

try:
    parse_grammar(grammar_text)
except RuntimeError as e:
    if 'unexpected end of input' in str(e):
        raise ValueError('Grammar literal is missing its closing quote') from None
    raise

When it happens

Trigger: A grammar rule like root ::= "unterminated — no closing quote before end of file; or an escape such as \u12 truncated mid-sequence leaving src empty; grammar files with the final line missing its trailing newline/quote after editor truncation.

Common situations: Truncated grammar files (partial download, editor cut-off); copy-paste that dropped the closing '"'; multi-line attempt inside a single-line literal where the quote after the line break is missing.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• expecting ')' at
• expecting ::= at
• expecting name at
• unknown escape at
• unknown hex char

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/1d9a2e49c634db0d. Report an issue: GitHub.

Appendix: source

Thrown at modules/grammar/grammar_utils.py:139

            return read_hex(src[2:4]), src[4:]
        elif esc == "u":
            return read_hex(src[2:6]), src[6:]
        elif esc == "U":
            return read_hex(src[2:10]), src[10:]
        elif esc in ('"', "[", "]", "\\", "-"):
            return esc, src[2:]
        elif esc == "r":
            return "\r", src[2:]
        elif esc == "n":
            return "\n", src[2:]
        elif esc == "t":
            return "\t", src[2:]
        elif esc == "\\":
            return "\\", src[2:]
        raise RuntimeError("unknown escape at " + src)
    elif src:
        return src[0], src[1:]
    raise RuntimeError("unexpected end of input")


def parse_sequence(state, src, rule_name, outbuf, is_nested):
    out_start_pos = len(outbuf)

    # sequence size, will be replaced at end when known
    outbuf.append(TO_BE_FILLED_MARKER)

    last_sym_start = len(outbuf)
    remaining_src = src
    while remaining_src:
        if remaining_src[0] == '"':  # literal string
            remaining_src = remaining_src[1:]
            last_sym_start = len(outbuf)
            while remaining_src[0] != '"':
                char, remaining_src = parse_char(remaining_src)

                # each char of a literal is encoded as a "range" of char - char

View on GitHub (pinned to ed888c71f2)