pypa/pip · error · TypeError

lex() argument must be a lexer instance, not a class

Error message

lex() argument must be a lexer instance, not a class

What it means

Raised as TypeError by pygments.lex() when the lexer argument is a Lexer *class* rather than an *instance*. lex() calls lexer.get_tokens(code); passing the class raises a TypeError from unbound-method access, which the heuristic catches and re-raises with this clearer message when the class subclasses RegexLexer.

Source

Thrown at src/pip/_vendor/pygments/__init__.py:47

__version__ = '2.20.0'
__docformat__ = 'restructuredtext'

__all__ = ['lex', 'format', 'highlight']


def lex(code, lexer):
    """
    Lex `code` with the `lexer` (must be a `Lexer` instance)
    and return an iterable of tokens. Currently, this only calls
    `lexer.get_tokens()`.
    """
    try:
        return lexer.get_tokens(code)
    except TypeError:
        # Heuristic to catch a common mistake.
        from pip._vendor.pygments.lexer import RegexLexer
        if isinstance(lexer, type) and issubclass(lexer, RegexLexer):
            raise TypeError('lex() argument must be a lexer instance, '
                            'not a class')
        raise


def format(tokens, formatter, outfile=None):  # pylint: disable=redefined-builtin
    """
    Format ``tokens`` (an iterable of tokens) with the formatter ``formatter``
    (a `Formatter` instance).

    If ``outfile`` is given and a valid file object (an object with a
    ``write`` method), the result will be written to it, otherwise it
    is returned as a string.
    """
    try:
        if not outfile:
            realoutfile = getattr(formatter, 'encoding', None) and BytesIO() or StringIO()
            formatter.format(tokens, realoutfile)
            return realoutfile.getvalue()

View on GitHub (pinned to d7d0d0a394)

Solutions

1. Instantiate the lexer: pass PythonLexer() (with parentheses) to lex().
2. Use pygments.lexers.get_lexer_by_name('python') which returns a ready instance.
3. Double-check that your variable holds an instance, not the class object.

Example fix

# before
from pygments import lex
from pygments.lexers import PythonLexer
tokens = lex(code, PythonLexer)  # TypeError

# after
tokens = lex(code, PythonLexer())

from pygments.lexer import RegexLexer
if isinstance(lexer, type) and issubclass(lexer, RegexLexer):
    raise TypeError('pass a lexer instance, not the class')
pygments.lex(code, lexer)

def is_lexer_instance(obj) -> bool:
    from pygments.lexer import Lexer
    return not isinstance(obj, type) and isinstance(obj, Lexer)

try:
    tokens = pygments.lex(code, lexer)
except TypeError as e:
    if 'must be a lexer instance' in str(e):
        tokens = pygments.lex(code, lexer())  # instantiate class
    raise

When it happens

Trigger: Calling pygments.lex(code, PythonLexer) (the class) instead of pygments.lex(code, PythonLexer()). The except-TypError branch detects issubclass(lexer, RegexLexer) and raises the helpful message.

Common situations: Forgetting to instantiate the lexer, copying example code that omitted parentheses, or passing get_lexer_by_name result incorrectly.

Related errors

• format() argument must be a formatter instance, not a class
• {self.__class__.__name__!r} used without bound function
• filter {filtername!r} not found
• No such group
• uncompilable regex {tdef[0]!r} in state {state!r} of {cls!r}

AI-assisted analysis of pypa/pip@d7d0d0a394 (2026-08-04). Data as JSON: /data/errors/477e0585e600b8ba.json. Report an issue: GitHub.