{"record":{"id":"45a8eaf59ba63a4c","repo":"jamiebuilds/the-super-tiny-compiler","slug":"token-type","errorCode":null,"errorMessage":"token.type","messagePattern":"token\\.type","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"the-super-tiny-compiler.js","lineNumber":672,"sourceCode":"        (token.type === 'paren' && token.value !== ')')\n      ) {\n        // we'll call the `walk` function which will return a `node` and we'll\n        // push it into our `node.params`.\n        node.params.push(walk());\n        token = tokens[current];\n      }\n\n      // Finally we will increment `current` one last time to skip the closing\n      // parenthesis.\n      current++;\n\n      // And return the node.\n      return node;\n    }\n\n    // Again, if we haven't recognized the token type by now we're going to\n    // throw an error.\n    throw new TypeError(token.type);\n  }\n\n  // Now, we're going to create our AST which will have a root which is a\n  // `Program` node.\n  let ast = {\n    type: 'Program',\n    body: [],\n  };\n\n  // And we're going to kickstart our `walk` function, pushing nodes to our\n  // `ast.body` array.\n  //\n  // The reason we are doing this inside a loop is because our program can have\n  // `CallExpression` after one another instead of being nested.\n  //\n  //   (add 2 2)\n  //   (subtract 4 2)\n  //","sourceCodeStart":654,"sourceCodeEnd":690,"githubUrl":"https://github.com/jamiebuilds/the-super-tiny-compiler/blob/d8d40130459d1537f6117a927947cd46c83182b0/the-super-tiny-compiler.js#L654-L690","documentation":"The parser's walk() function throws the token's type string when it encounters a token type it doesn't know how to turn into an AST node. The supported token types are 'number', 'string', 'paren', and 'name'; anything else reaches the default throw at the end of the loop. Because the message is just the token type (e.g. 'operator'), it identifies which token type was unrecognized.","triggerScenarios":"Calling parser(tokens) with a token array whose entries have type values outside {'number','string','paren','name'}, or where a 'paren' token's value is neither '(' nor ')'. This happens when tokens are hand-crafted or produced by a modified/buggy tokenizer rather than the library's own tokenizer().","commonSituations":"Passing hand-constructed tokens or output of a custom tokenizer into parser(). Mixing versions of the compiler where the tokenizer emits new token types (e.g. an added 'operator' token) but the unmodified parser is used. Mutating tokens between tokenizer() and parser().","solutions":["Feed parser() only the exact array returned by tokenizer(); do not construct tokens by hand.","If you extended the tokenizer with new token types, add matching branches in walk() before the final throw.","Validate token types against {'number','string','paren','name'} before calling parser().","Check that a modified tokenizer still wraps expressions in paren tokens with value '(' — walk() descends only on that exact value."],"exampleFix":"// before\nparser([{ type: 'operator', value: '+' }]); // throws TypeError: operator\n\n// after\nconst tokens = tokenizer('(add 1 2)');\nparser(tokens);","handlingStrategy":"type-guard","validationCode":"const KNOWN_TOKEN_TYPES = new Set(['number', 'string', 'paren', 'name']);\nfunction tokensAreValid(tokens) {\n  return Array.isArray(tokens) && tokens.every(t => t && typeof t.type === 'string' && KNOWN_TOKEN_TYPES.has(t.type));\n}\nif (!tokensAreValid(tokens)) throw new Error('Unsupported token type');\nparser(tokens);","typeGuard":"function isKnownToken(t) {\n  return t != null && ['number','string','paren','name'].includes(t.type);\n}","tryCatchPattern":"try { ast = parser(tokens); } catch (e) { if (e instanceof TypeError && ['number','string','paren','name'].indexOf(e.message) === -1) { /* unknown token type: e.message */ } else throw e; }","preventionTips":["Always pass parser() the array returned directly by tokenizer(); never hand-build tokens.","If you add token types to the tokenizer, add matching cases in walk() in the same change.","Freeze/clone tokens between stages to avoid accidental mutation of type fields."],"tags":["parser","ast","invalid-token","type-error"],"backgroundTag":"unrecognized-token-type","analyzedSha":"d8d40130459d1537f6117a927947cd46c83182b0","analyzedAt":"2026-08-28T20:32:54.726Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}