jamiebuilds/the-super-tiny-compiler · error · TypeError

token.type

Error message

token.type

What it means

The parser's walk() function throws the token's type string when it encounters a token type it doesn't know how to turn into an AST node. The supported token types are 'number', 'string', 'paren', and 'name'; anything else reaches the default throw at the end of the loop. Because the message is just the token type (e.g. 'operator'), it identifies which token type was unrecognized.

Source

Thrown at the-super-tiny-compiler.js:672

        (token.type === 'paren' && token.value !== ')')
      ) {
        // we'll call the `walk` function which will return a `node` and we'll
        // push it into our `node.params`.
        node.params.push(walk());
        token = tokens[current];
      }

      // Finally we will increment `current` one last time to skip the closing
      // parenthesis.
      current++;

      // And return the node.
      return node;
    }

    // Again, if we haven't recognized the token type by now we're going to
    // throw an error.
    throw new TypeError(token.type);
  }

  // Now, we're going to create our AST which will have a root which is a
  // `Program` node.
  let ast = {
    type: 'Program',
    body: [],
  };

  // And we're going to kickstart our `walk` function, pushing nodes to our
  // `ast.body` array.
  //
  // The reason we are doing this inside a loop is because our program can have
  // `CallExpression` after one another instead of being nested.
  //
  //   (add 2 2)
  //   (subtract 4 2)
  //

View on GitHub (pinned to d8d4013045)

Solutions

1. Feed parser() only the exact array returned by tokenizer(); do not construct tokens by hand.
2. If you extended the tokenizer with new token types, add matching branches in walk() before the final throw.
3. Validate token types against {'number','string','paren','name'} before calling parser().
4. Check that a modified tokenizer still wraps expressions in paren tokens with value '(' — walk() descends only on that exact value.

Example fix

// before
parser([{ type: 'operator', value: '+' }]); // throws TypeError: operator

// after
const tokens = tokenizer('(add 1 2)');
parser(tokens);

const KNOWN_TOKEN_TYPES = new Set(['number', 'string', 'paren', 'name']);
function tokensAreValid(tokens) {
  return Array.isArray(tokens) && tokens.every(t => t && typeof t.type === 'string' && KNOWN_TOKEN_TYPES.has(t.type));
}
if (!tokensAreValid(tokens)) throw new Error('Unsupported token type');
parser(tokens);

function isKnownToken(t) {
  return t != null && ['number','string','paren','name'].includes(t.type);
}

try { ast = parser(tokens); } catch (e) { if (e instanceof TypeError && ['number','string','paren','name'].indexOf(e.message) === -1) { /* unknown token type: e.message */ } else throw e; }

When it happens

Trigger: Calling parser(tokens) with a token array whose entries have type values outside {'number','string','paren','name'}, or where a 'paren' token's value is neither '(' nor ')'. This happens when tokens are hand-crafted or produced by a modified/buggy tokenizer rather than the library's own tokenizer().

Common situations: Passing hand-constructed tokens or output of a custom tokenizer into parser(). Mixing versions of the compiler where the tokenizer emits new token types (e.g. an added 'operator' token) but the unmodified parser is used. Mutating tokens between tokenizer() and parser().

Related errors

• I dont know what this character is: ' + char
• node.type

AI-assisted analysis of jamiebuilds/the-super-tiny-compiler@d8d4013045 (2026-08-28). Data as JSON: /api/errors/45a8eaf59ba63a4c. Report an issue: GitHub.