What causes "Invalid input type: {type(text)}" in karpathy/nanochat?

Convert input to str or list[str] before calling encode: `tokenizer.encode(text.item())` for 0-d numpy, `.tolist()` for tensors of ids is a decode-side operation.

karpathy/nanochat · error · ValueError

Invalid input type: {type(text)}

Error message

Invalid input type: {type(text)}

What it means

Tokenizer.encode accepts exactly two input shapes: a single string (returns list[int]) or a list of strings (returns list[list[int]] via encode_ordinary_batch). Anything else — int, None, bytes, a numpy array, a torch.Tensor, a nested list — reaches the else and raises ValueError with the offending type.

Source

Thrown at nanochat/tokenizer.py:119

        if append is not None:
            append_id = append if isinstance(append, int) else self.encode_special(append)

        if isinstance(text, str):
            ids = self.enc.encode_ordinary(text)
            if prepend is not None:
                ids.insert(0, prepend_id) # TODO: slightly inefficient here? :( hmm
            if append is not None:
                ids.append(append_id)
        elif isinstance(text, list):
            ids = self.enc.encode_ordinary_batch(text, num_threads=num_threads)
            if prepend is not None:
                for ids_row in ids:
                    ids_row.insert(0, prepend_id) # TODO: same
            if append is not None:
                for ids_row in ids:
                    ids_row.append(append_id)
        else:
            raise ValueError(f"Invalid input type: {type(text)}")

        return ids

    def __call__(self, *args, **kwargs):
        return self.encode(*args, **kwargs)

    def decode(self, ids):
        return self.enc.decode(ids)

    def decode_single_token_bytes(self, token_id):
        return self.enc.decode_single_token_bytes(token_id)

    def save(self, tokenizer_dir):
        # save the encoding object to disk
        os.makedirs(tokenizer_dir, exist_ok=True)
        pickle_path = os.path.join(tokenizer_dir, "tokenizer.pkl")
        with open(pickle_path, "wb") as f:
            pickle.dump(self.enc, f)

View on GitHub (pinned to 92d63d4e8b)

Solutions

Convert input to str or list[str] before calling encode: `tokenizer.encode(text.item())` for 0-d numpy, `.tolist()` for tensors of ids is a decode-side operation.
If input is already token ids, use `tokenizer.decode(ids)` instead of encode.
For batch mode, ensure every element is a str: `all(isinstance(t, str) for t in texts)`.

Example fix

# before
ids = tokenizer.encode(example["tokens"])  # list[int], not list[str]

# after
text = tokenizer.decode(example["tokens"])
ids = tokenizer.encode(text)

Defensive patterns

Strategy: type-guard

Validate before calling

if not isinstance(text, (str, list)) or (isinstance(text, list) and not all(isinstance(t, str) for t in text)):
    raise TypeError(f"encode expects str or list[str], got {type(text).__name__}")

Type guard

def is_valid_encode_input(text) -> bool:
    if isinstance(text, str):
        return True
    return isinstance(text, list) and all(isinstance(t, str) for t in text)

Try / catch

try:
    ids = tokenizer.encode(text)
except ValueError:
    text = str(text) if not isinstance(text, list) else [str(t) for t in text]
    ids = tokenizer.encode(text)

Prevention

Coerce numpy/tensor scalars to Python str before encoding.
Remember ids go to decode(), strings go to encode() — never mix directions.
For batch inputs, verify every element is str.

When it happens

Trigger: Calling tokenizer.encode(123), tokenizer.encode(None), tokenizer.encode([1,2,3]) (a list of ints, not strings), or passing a torch tensor/numpy array of text instead of Python strings.

Common situations: Feeding already-tokenized ids back into encode; passing a numpy array of strings from a data pipeline; passing None from a missing dataset field; mixing up encode and decode directions.

Related errors

AI-assisted analysis of karpathy/nanochat@92d63d4e8b (2026-08-15). Data as JSON: /api/errors/dbd8f19715f1ded1. Report an issue: GitHub.