{"record":{"id":"edb0e3e732d984b9","repo":"jamiebuilds/the-super-tiny-compiler","slug":"i-dont-know-what-this-character-is-char","errorCode":null,"errorMessage":"I dont know what this character is: ' + char","messagePattern":"I dont know what this character is: ' \\+ char","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"the-super-tiny-compiler.js","lineNumber":533,"sourceCode":"    if (LETTERS.test(char)) {\n      let value = '';\n\n      // Again we're just going to loop through all the letters pushing them to\n      // a value.\n      while (LETTERS.test(char)) {\n        value += char;\n        char = input[++current];\n      }\n\n      // And pushing that value as a token with the type `name` and continuing.\n      tokens.push({ type: 'name', value });\n\n      continue;\n    }\n\n    // Finally if we have not matched a character by now, we're going to throw\n    // an error and completely exit.\n    throw new TypeError('I dont know what this character is: ' + char);\n  }\n\n  // Then at the end of our `tokenizer` we simply return the tokens array.\n  return tokens;\n}\n\n/**\n * ============================================================================\n *                                 ヽ/❀o ل͜ o\\ﾉ\n *                                THE PARSER!!!\n * ============================================================================\n */\n\n/**\n * For our parser we're going to take our array of tokens and turn it into an\n * AST.\n *\n *   [{ type: 'paren', value: '(' }, ...]   =>   { type: 'Program', body: [...] }","sourceCodeStart":515,"sourceCodeEnd":551,"githubUrl":"https://github.com/jamiebuilds/the-super-tiny-compiler/blob/d8d40130459d1537f6117a927947cd46c83182b0/the-super-tiny-compiler.js#L515-L551","documentation":"This error is thrown by the tokenizer when it encounters a character that does not match any token pattern: whitespace, semicolon, parenthesis, number, string, or name. It is the tokenizer's catch-all guard meaning the input source contains a character the compiler's grammar does not recognize. The character is appended to the message so you can identify the offending byte.","triggerScenarios":"Calling tokenizer(input) on a string containing characters outside the supported grammar, e.g. commas, operators (+ - * / = < >), brackets [ ] { }, quotes with unusual escaping, comments, or non-ASCII characters. Any one unrecognized char aborts tokenization immediately.","commonSituations":"Assuming the super-tiny-compiler is a general JS parser and feeding it arbitrary JavaScript (e.g. '(add 1, 2)' with a comma, or 'x = 5' with an equals sign). It only supports the tiny Lisp-like grammar: parens, names, numbers, double-quoted strings, whitespace, and semicolons.","solutions":["Remove or replace the offending character shown in the message — the compiler only accepts parens, names, numbers, double-quoted strings, whitespace, and semicolons.","Strip or sanitize input before tokenizing (e.g. remove commas/operators) if you control the input format.","If you need a real JS parser, switch to a full parser such as acorn, espree, or @babel/parser."],"exampleFix":"// before (comma is unsupported)\ntokenizer('(add 1, 2)');\n// throws TypeError: I dont know what this character is: ,\n\n// after (whitespace-separated arguments)\ntokenizer('(add 1 2)');","handlingStrategy":"validation","validationCode":"const VALID = /^[\\s();()a-zA-Z0-9\"]+$/; // rough grammar surface\nfunction isTokenizable(input) {\n  return typeof input === 'string' && !/[^\\s();()a-zA-Z0-9\"]/.test(input.replace(/\"[^\"]*\"/g, '\"\"'));\n}\nif (!isTokenizable(src)) throw new Error('Input contains unsupported characters');\nconst tokens = tokenizer(src);","typeGuard":"function isTokenizableSource(input) {\n  return typeof input === 'string' && /^[\\s();()a-zA-Z0-9\"]*$/.test(input.replace(/\"[^\"]*\"/g, '\"\"'));\n}","tryCatchPattern":"try { tokens = tokenizer(src); } catch (e) { if (e instanceof TypeError && /I dont know what this character is/.test(e.message)) { /* report offending char: e.message.split(': ')[1] */ } else throw e; }","preventionTips":["Restrict input to the tiny grammar: parens, names, numbers, double-quoted strings, whitespace, semicolons.","Sanitize/strip commas, operators, and comments before tokenizing.","Write property tests asserting tokenizer(input) never throws for accepted inputs."],"tags":["tokenizer","syntax-error","invalid-input","parser"],"backgroundTag":"unexpected-character-parse-error","analyzedSha":"d8d40130459d1537f6117a927947cd46c83182b0","analyzedAt":"2026-08-28T20:32:54.726Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}