pypa/pip · error · ValueError

payload in an invalid encoding

Error message

payload in an invalid encoding

What it means

Raised as ValueError('payload in an invalid encoding') by _get_payload in packaging.metadata when the METADATA body bytes cannot be decoded as strict UTF-8. packaging assumes PEP 621/643 metadata is UTF-8; any non-UTF-8 byte sequence is rejected.

Source

Thrown at src/pip/_vendor/packaging/metadata.py:238

def _get_payload(msg: email.message.Message, source: bytes | str) -> str:
    """Get the body of the message."""
    # If our source is a str, then our caller has managed encodings for us,
    # and we don't need to deal with it.
    if isinstance(source, str):
        payload = msg.get_payload()
        assert isinstance(payload, str)
        return payload
    # If our source is a bytes, then we're managing the encoding and we need
    # to deal with it.
    else:
        bpayload = msg.get_payload(decode=True)
        assert isinstance(bpayload, bytes)
        try:
            return bpayload.decode("utf8", "strict")
        except UnicodeDecodeError as exc:
            raise ValueError("payload in an invalid encoding") from exc


# The various parse_FORMAT functions here are intended to be as lenient as
# possible in their parsing, while still returning a correctly typed
# RawMetadata.
#
# To aid in this, we also generally want to do as little touching of the
# data as possible, except where there are possibly some historic holdovers
# that make valid data awkward to work with.
#
# While this is a lower level, intermediate format than our ``Metadata``
# class, some light touch ups can make a massive difference in usability.

# Map METADATA fields to RawMetadata.
_EMAIL_TO_RAW_MAPPING = {
    "author": "author",
    "author-email": "author_email",
    "classifier": "classifiers",

View on GitHub (pinned to d7d0d0a394)

Solutions

1. Re-save the METADATA file as UTF-8 (e.g. iconv -f latin1 -t utf-8).
2. If you must pass str, decode the bytes yourself with the correct encoding first and pass the str to from_email.
3. Strip or fix the offending non-UTF-8 bytes before parsing.
4. Use twine check / build to validate the sdist metadata before consumption.

Example fix

# before
from_email(raw_latin1_bytes)
# after
from_email(raw_latin1_bytes.decode('latin-1'))

def is_utf8(b: bytes) -> bool:
    try:
        b.decode('utf-8', 'strict')
        return True
    except UnicodeDecodeError:
        return False

try:
    md = Metadata.from_email(raw_bytes)
except ValueError as e:
    if 'invalid encoding' in str(e):
        md = Metadata.from_email(raw_bytes.decode('latin-1'))

When it happens

Trigger: Passing bytes source to from_email where the message body contains Latin-1, CP1252, or CJK bytes; a PKG-INFO / METADATA file saved as Latin-1 by an old tool; an XML/encoding declaration in the body that contradicts the actual bytes.

Common situations: Reading legacy sdists whose METADATA was authored pre-UTF-8; files edited on Windows in CP1252; mojibake introduced by a misconfigured build backend.

Related errors

• Error decoding metadata for {wheel}: {e} in {filename} file
• Error decoding metadata for {self._wheel_name}: {e} in {name
• error decoding {path!r}: {e!r}
• {name} has an invalid wheel, {e}
• multiple .dist-info directories found: {}

AI-assisted analysis of pypa/pip@d7d0d0a394 (2026-08-04). Data as JSON: /data/errors/e96505a09dc333b8.json. Report an issue: GitHub.