oobabooga/textgen · warning · RuntimeError

unknown hex char {c}

Error message

unknown hex char {c}

What it means

Raised by hex_to_int (modules/grammar/grammar_utils.py:63) — part of the GBNF grammar compiler used for llama.cpp constrained generation (--grammar / grammar_string). While reading \xHH or \uHHHH escapes via read_hex, a character outside [0-9a-fA-F] appears, so the escape cannot be decoded.

Source

Thrown at modules/grammar/grammar_utils.py:63

    return state.symbol_ids[src]


def generate_symbol_id(state, base_name):
    next_id = len(state.symbol_ids)
    state.symbol_ids[base_name + "_" + str(next_id)] = next_id
    return next_id


def is_word_char(c):
    return c.isalnum() or c == "-" or c == "_"


def hex_to_int(c):
    if c.isdigit():
        return int(c)
    elif "a" <= c.lower() <= "f":
        return ord(c.lower()) - ord("a") + 10
    raise RuntimeError("unknown hex char " + c)


def remove_leading_white_space(src, newline_ok):
    """
    Skips over whitespace and comments in the input string.
    This function processes the input string, skipping over any spaces, tabs,
    and content following a '#' character, which denotes a comment. The parsing
    of a comment continues until the end of the line (denoted by newline characters
    '\r' or '\n'). If the 'newline_ok' parameter is set to False, the function
    will stop processing and return the remaining string upon encountering a
    newline character, otherwise it will skip over newline characters as well.
    Parameters:
    src (str): The input string to be processed.
    newline_ok (bool): A flag indicating whether encountering a newline character
                       should stop the parsing (False) or if it should be skipped (True).
    Returns:
    str: The remaining portion of the input string after skipping whitespace and comments.
    """

View on GitHub (pinned to ed888c71f2)

Solutions

1. Fix the escape: every \x must be followed by exactly 2 hex digits, \u by 4, \U by 8.
2. If you meant a literal character, write it directly or use \r/\n/\t named escapes.
3. Lint the grammar with a quick python check that all \\x[0-9a-fA-F]{2} patterns are well-formed before submitting.

Example fix

// before
root ::= "a\xG1b"

// after
root ::= "a\x41b"   // \x41 = 'A'

import re

HEX_OK = re.compile(r'\\(?:x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4}|U[0-9a-fA-F]{8})')
BAD_ESCAPE = re.compile(r'\\(?:x|u|U)(?![0-9a-fA-F])')

def grammar_escapes_ok(g: str) -> bool:
    return not BAD_ESCAPE.search(g)

def is_valid_grammar_text(g: str) -> bool:
    return bool(g) and grammar_escapes_ok(g)

from modules.grammar.grammar_utils import parse_grammar
try:
    parse_grammar(grammar_text)
except RuntimeError as e:
    raise ValueError(f'Grammar rejected: {e}') from None

When it happens

Trigger: Submitting a grammar containing a malformed hex escape, e.g. \xGG, \u00ZZ, or a truncated escape like \x1 (next char is '"'). Used via modules/grammar when passing grammar_string to a llama.cpp-backed model.

Common situations: Hand-written grammars with typos in escapes; grammars generated by templating that leaves placeholders (e.g. \x{ID}); copy-paste from docs where the escape got mangled; confusing decimal (\x20 vs \32) notation.

Related errors

• expecting name at {src}
• unknown escape at {src}
• unexpected end of input
• expecting ')' at {remaining_src}
• expecting ::= at {remaining_src}

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/f87fe64bd73c1377. Report an issue: GitHub.