{"record":{"id":"f87fe64bd73c1377","repo":"oobabooga/textgen","slug":"unknown-hex-char-c","errorCode":null,"errorMessage":"unknown hex char {c}","messagePattern":"unknown hex char (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"warning","filePath":"modules/grammar/grammar_utils.py","lineNumber":63,"sourceCode":"    return state.symbol_ids[src]\n\n\ndef generate_symbol_id(state, base_name):\n    next_id = len(state.symbol_ids)\n    state.symbol_ids[base_name + \"_\" + str(next_id)] = next_id\n    return next_id\n\n\ndef is_word_char(c):\n    return c.isalnum() or c == \"-\" or c == \"_\"\n\n\ndef hex_to_int(c):\n    if c.isdigit():\n        return int(c)\n    elif \"a\" <= c.lower() <= \"f\":\n        return ord(c.lower()) - ord(\"a\") + 10\n    raise RuntimeError(\"unknown hex char \" + c)\n\n\ndef remove_leading_white_space(src, newline_ok):\n    \"\"\"\n    Skips over whitespace and comments in the input string.\n    This function processes the input string, skipping over any spaces, tabs,\n    and content following a '#' character, which denotes a comment. The parsing\n    of a comment continues until the end of the line (denoted by newline characters\n    '\\r' or '\\n'). If the 'newline_ok' parameter is set to False, the function\n    will stop processing and return the remaining string upon encountering a\n    newline character, otherwise it will skip over newline characters as well.\n    Parameters:\n    src (str): The input string to be processed.\n    newline_ok (bool): A flag indicating whether encountering a newline character\n                       should stop the parsing (False) or if it should be skipped (True).\n    Returns:\n    str: The remaining portion of the input string after skipping whitespace and comments.\n    \"\"\"","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/oobabooga/textgen/blob/ed888c71f221df552750e1834b3654abab8ae345/modules/grammar/grammar_utils.py#L45-L81","documentation":"Raised by hex_to_int (modules/grammar/grammar_utils.py:63) — part of the GBNF grammar compiler used for llama.cpp constrained generation (--grammar / grammar_string). While reading \\xHH or \\uHHHH escapes via read_hex, a character outside [0-9a-fA-F] appears, so the escape cannot be decoded.","triggerScenarios":"Submitting a grammar containing a malformed hex escape, e.g. \\xGG, \\u00ZZ, or a truncated escape like \\x1 (next char is '\"'). Used via modules/grammar when passing grammar_string to a llama.cpp-backed model.","commonSituations":"Hand-written grammars with typos in escapes; grammars generated by templating that leaves placeholders (e.g. \\x{ID}); copy-paste from docs where the escape got mangled; confusing decimal (\\x20 vs \\32) notation.","solutions":["Fix the escape: every \\x must be followed by exactly 2 hex digits, \\u by 4, \\U by 8.","If you meant a literal character, write it directly or use \\r/\\n/\\t named escapes.","Lint the grammar with a quick python check that all \\\\x[0-9a-fA-F]{2} patterns are well-formed before submitting."],"exampleFix":"// before\nroot ::= \"a\\xG1b\"\n\n// after\nroot ::= \"a\\x41b\"   // \\x41 = 'A'\n","handlingStrategy":"validation","validationCode":"import re\n\nHEX_OK = re.compile(r'\\\\(?:x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4}|U[0-9a-fA-F]{8})')\nBAD_ESCAPE = re.compile(r'\\\\(?:x|u|U)(?![0-9a-fA-F])')\n\ndef grammar_escapes_ok(g: str) -> bool:\n    return not BAD_ESCAPE.search(g)\n","typeGuard":"def is_valid_grammar_text(g: str) -> bool:\n    return bool(g) and grammar_escapes_ok(g)\n","tryCatchPattern":"from modules.grammar.grammar_utils import parse_grammar\ntry:\n    parse_grammar(grammar_text)\nexcept RuntimeError as e:\n    raise ValueError(f'Grammar rejected: {e}') from None\n","preventionTips":["Generate hex escapes programmatically (e.g. f'\\\\x{ord(c):02x}') instead of hand-typing them.","Compile/parse grammars once at startup and cache the parsed form.","Write grammar files as UTF-8 and avoid editor auto-escaping."],"tags":["grammar","gbnf","validation","llama-cpp"],"backgroundTag":null,"analyzedSha":"ed888c71f221df552750e1834b3654abab8ae345","analyzedAt":"2026-08-15T05:24:21.000Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}