nodejs/node · error · ValueError

payload in an invalid encoding

Error message

payload in an invalid encoding

What it means

Raised by packaging.metadata.parse_email when the metadata source is bytes and the email message body cannot be decoded as strict UTF-8. The library treats a str source as already encoding-managed by the caller, but for bytes it must decode the payload itself and rejects anything that is not valid UTF-8. It surfaces as a ValueError wrapping a UnicodeDecodeError.

Source

Thrown at tools/gyp/pylib/packaging/metadata.py:230

    return urls


def _get_payload(msg: email.message.Message, source: Union[bytes, str]) -> str:
    """Get the body of the message."""
    # If our source is a str, then our caller has managed encodings for us,
    # and we don't need to deal with it.
    if isinstance(source, str):
        payload: str = msg.get_payload()
        return payload
    # If our source is a bytes, then we're managing the encoding and we need
    # to deal with it.
    else:
        bpayload: bytes = msg.get_payload(decode=True)
        try:
            return bpayload.decode("utf8", "strict")
        except UnicodeDecodeError:
            raise ValueError("payload in an invalid encoding")


# The various parse_FORMAT functions here are intended to be as lenient as
# possible in their parsing, while still returning a correctly typed
# RawMetadata.
#
# To aid in this, we also generally want to do as little touching of the
# data as possible, except where there are possibly some historic holdovers
# that make valid data awkward to work with.
#
# While this is a lower level, intermediate format than our ``Metadata``
# class, some light touch ups can make a massive difference in usability.

# Map METADATA fields to RawMetadata.
_EMAIL_TO_RAW_MAPPING = {
    "author": "author",
    "author-email": "author_email",
    "classifier": "classifiers",

View on GitHub (pinned to 1b2de5e052)

Solutions

1. Decode the bytes yourself with the charset declared by the email message (msg.get_content_charset()) and pass a str to from_email so the library skips its internal decode.
2. If the file is genuinely UTF-8 but corrupted, re-fetch or rebuild the source distribution.
3. As a last resort, decode with errors='replace' to inspect the metadata, then fix the upstream producer.

Example fix

# before
with open(path, 'rb') as f:
    data = f.read()
meta = Metadata.from_email(data)

# after
with open(path, 'r', encoding='utf-8') as f:
    data = f.read()
meta = Metadata.from_email(data)

def safe_from_email(data: bytes):
    import email
    msg = email.message_from_bytes(data)
    charset = msg.get_content_charset() or 'utf-8'
    try:
        text = data.decode(charset)
    except (LookupError, UnicodeDecodeError):
        text = data.decode('utf-8', errors='replace')
    return Metadata.from_email(text)

def is_utf8_decodable(b: bytes) -> bool:
    try:
        b.decode('utf-8', 'strict')
        return True
    except UnicodeDecodeError:
        return False

try:
    meta = Metadata.from_email(data)
except ValueError as e:
    if 'invalid encoding' in str(e):
        # decode with declared charset and retry, or report
        pass

When it happens

Trigger: Calling Metadata.from_email(some_bytes) or parse_email(some_bytes) where the bytes payload contains non-UTF-8 sequences (e.g. Latin-1, CP1252, or a CTE like base64 that decodes to a non-UTF-8 charset). Only the bytes branch exercises decode('utf8','strict'); a str input never hits this path.

Common situations: Reading a METADATA/PKG-INFO file in binary mode from a sdist built on a non-UTF-8 locale, a corrupted download, or a file that declares a charset other than utf-8 in its MIME headers but whose decoded bytes are not UTF-8. Also seen when upstream packaging produced a file with a legacy 8-bit encoding.

Related errors

• duplicate labels in project urls
• invalid metadata
• unparsed
• invalid or unparsed metadata
• key must be ascii string

AI-assisted analysis of nodejs/node@1b2de5e052 (2026-08-13). Data as JSON: /api/errors/ed44bf79f31fb19d. Report an issue: GitHub.