What causes "Could not convert '{name}' to a valid Python identifier." in pandas-dev/pandas?

Rename the column to a clean identifier before querying: df.columns = df.columns.str.replace(r'[^A-Za-z0-9_]', '_', regex=True).

pandas-dev/pandas · error · SyntaxError

Could not convert '{name}' to a valid Python identifier.

Error message

Could not convert '{name}' to a valid Python identifier.

What it means

Raised by create_valid_python_identifier in pandas.core.computation.parsing when a backtick-quoted column name (or any identifier fed through the cleaner) cannot be transformed into a legal Python identifier. The function first tries name.isidentifier(), then escapes non-ASCII via backslashreplace, maps special characters using tokenize.EXACT_TOKEN_TYPES plus an explicit table (space, ?, !, $, quotes, etc.), and prefixes the result with 'BACKTICK_QUOTED_STRING_'. If the final string still fails str.isidentifier(), it raises SyntaxError. This is the machinery behind `df.query('`weird col` > 0')`.

Source

Thrown at pandas/core/computation/parsing.py:77

        {
            " ": "_",
            "?": "_QUESTIONMARK_",
            "!": "_EXCLAMATIONMARK_",
            "$": "_DOLLARSIGN_",
            "€": "_EUROSIGN_",
            "°": "_DEGREESIGN_",
            "'": "_SINGLEQUOTE_",
            '"': "_DOUBLEQUOTE_",
            "#": "_HASH_",
            "`": "_BACKTICK_",
        }
    )

    name = "".join([special_characters_replacements.get(char, char) for char in name])
    name = f"BACKTICK_QUOTED_STRING_{name}"

    if not name.isidentifier():
        raise SyntaxError(f"Could not convert '{name}' to a valid Python identifier.")

    return name


def clean_backtick_quoted_toks(tok: tuple[int, str]) -> tuple[int, str]:
    """
    Clean up a column name if surrounded by backticks.

    Backtick quoted string are indicated by a certain tokval value. If a string
    is a backtick quoted token it will processed by
    :func:`_create_valid_python_identifier` so that the parser can find this
    string when the query is executed.
    In this case the tok will get the NAME tokval.

    Parameters
    ----------
    tok : tuple of int, str
        ints correspond to the all caps constants in the tokenize module

View on GitHub (pinned to 71959b8cb9)

Solutions

Rename the column to a clean identifier before querying: df.columns = df.columns.str.replace(r'[^A-Za-z0-9_]', '_', regex=True).
Avoid backtick syntax and select with boolean masks directly: df[df['weird col'] > 0].
Simplify the column name to ASCII alphanumerics + underscore, then re-issue the query.
Inspect the actual column name (print(repr(col))) to find the offending character.

Example fix

# before
import pandas as pd
df = pd.DataFrame({'a b\x00c': [1, 2]})
df.query('`a b\x00c` > 1')  # SyntaxError: Could not convert ...

# after (rename)
df.columns = ['abc']
df.query('abc > 1')
# or skip query syntax:
df[df.iloc[:, 0] > 1]

Defensive patterns

Strategy: validation

Validate before calling

import re

def safe_query_column(name: str) -> str:
    cleaned = re.sub(r'[^A-Za-z0-9_]', '_', name)
    if not cleaned or cleaned[0].isdigit():
        cleaned = f'col_{cleaned}'
    if not cleaned.isidentifier():
        raise SyntaxError(f'column {name!r} cannot be used in query()')
    return cleaned

Type guard

from keyword import iskeyword

def is_query_safe_column(name: str) -> bool:
    return isinstance(name, str) and name.isidentifier() and not iskeyword(name)

Try / catch

try:
    df.query(f'`{col}` > 0')
except SyntaxError as e:
    if 'Could not convert' in str(e):
        # rename column or use boolean mask
        df[df[col] > 0]
    raise

Prevention

Sanitize column names to valid identifiers before storing data.
Prefer boolean masks df[df[col] > 0] over query() for messy column names.
Inspect problematic names with print(repr(col)) to find control chars.

When it happens

Trigger: Using a backtick-quoted column name whose content, after escaping, still violates identifier rules - e.g. a name consisting solely of characters that map to nothing legal, an empty backtick pair, or names with control characters / leading digits combined with disallowed chars that the escape table cannot rescue.

Common situations: DataFrames with column names containing unusual punctuation or control characters; programmatic query construction where the column name is dynamic; CSVs whose headers contain reserved symbols. Most ordinary spaces/punctuation are handled, so this fires only for genuinely pathological names.

Related errors

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/db1e87b46b052672. Report an issue: GitHub.