pola-rs/polars · error · ValueError

`encoding` must be one of {{'hex', 'base64'}}, got {encoding

Error message

`encoding` must be one of {{'hex', 'base64'}}, got {encoding!r}

What it means

Expr.str.decode is a transfer-encoding decoder limited to exactly 'hex' and 'base64'; the Python layer validates encoding before dispatch and raises ValueError for anything else. It is not a general text decoder — UTF-8/ASCII/latin-1 handling lives elsewhere (casting or UDFs), not in this namespace.

Source

Thrown at py-polars/src/polars/expr/string.py:1447

        >>> df.with_columns(pl.col("color").str.decode("hex").alias("decoded"))
        shape: (3, 2)
        ┌────────┬─────────────────┐
        │ color  ┆ decoded         │
        │ ---    ┆ ---             │
        │ str    ┆ binary          │
        ╞════════╪═════════════════╡
        │ 000000 ┆ b"\x00\x00\x00" │
        │ ffff00 ┆ b"\xff\xff\x00" │
        │ 0000ff ┆ b"\x00\x00\xff" │
        └────────┴─────────────────┘
        """
        if encoding == "hex":
            return wrap_expr(self._pyexpr.str_hex_decode(strict))
        elif encoding == "base64":
            return wrap_expr(self._pyexpr.str_base64_decode(strict))
        else:
            msg = f"`encoding` must be one of {{'hex', 'base64'}}, got {encoding!r}"
            raise ValueError(msg)

    def encode(self, encoding: TransferEncoding) -> Expr:
        """
        Encode values using the provided encoding.

        Parameters
        ----------
        encoding : {'hex', 'base64'}
            The encoding to use.

        .. engine-support:: in-memory, streaming, distributed

        Returns
        -------
        Expr
            Expression of data type :class:`String`.

        Examples

View on GitHub (pinned to df599052da)

Solutions

1. Use 'hex' or 'base64' exactly (lowercase, no surrounding whitespace)
2. Normalize untrusted input up front: encoding = encoding.strip().lower() and validate against the allowed set
3. For text encodings, cast instead (.cast(pl.String) on binary data) or decode inside map_batches with a Python UDF

Example fix

# before
pl.col('raw').str.decode('utf-8')

# after (hex/base64 payloads)
pl.col('raw').str.decode('base64')

# for actual text decoding, cast:
pl.col('raw').cast(pl.String)

ALLOWED = {'hex', 'base64'}
encoding = (encoding or '').strip().lower()
if encoding not in ALLOWED:
    raise ValueError(f'unsupported transfer encoding: {encoding!r}')
out = df.select(pl.col('raw').str.decode(encoding))

def is_transfer_encoding(x: object) -> bool:
    return isinstance(x, str) and x in {'hex', 'base64'}

try:
    expr = pl.col('raw').str.decode(encoding)
except ValueError as e:
    if 'must be one of' in str(e):
        expr = pl.col('raw').str.decode('base64')  # chosen default
    else:
        raise

When it happens

Trigger: .str.decode('utf-8'), .str.decode('ascii'), .str.decode('Hex') (wrong case), .str.decode('hex ') (whitespace), or an encoding variable sourced from user input/config that was never validated.

Common situations: Assuming str.decode mirrors Python's bytes.decode; config- or CLI-driven encoding names; copy-paste from code that encoded with .str.encode('base64') but decoded with a text codec name; typos and case mismatches.

Related errors

• {arg_name}="{arg}" should be a single byte character or empt
• {arg_name}="{arg}" should be a single byte character, but is
• unsupported encoding {encoding} for hf:// paths
• index positions should be smaller than 2^32
• index positions should be greater than or equal to -2^32

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/3f659c2561b245cd. Report an issue: GitHub.