aio-libs/aiohttp · error · ValueError

data cannot be decoded with %s encoding

Error message

data cannot be decoded with %s encoding

What it means

Raised by BodyPartReader.form() when the body part's bytes cannot be decoded using the charset resolved from the Content-Type header (or the encoding argument) before being parsed as application/x-www-form-urlencoded data. The underlying UnicodeDecodeError from bytes.decode() is converted into a ValueError so callers get a clear, single failure point. This guards the parse_qsl step, which would otherwise produce garbage or crash on mojibake.

Source

Thrown at aiohttp/multipart.py:517

        data = await self.read(decode=True)
        if not data:
            return None
        encoding = encoding or self.get_charset(default="utf-8")
        return cast(dict[str, Any], json.loads(data.decode(encoding)))

    async def form(self, *, encoding: str | None = None) -> list[tuple[str, str]]:
        """Like read(), but assumes that body parts contain form urlencoded data."""
        data = await self.read(decode=True)
        if not data:
            return []
        if encoding is not None:
            real_encoding = encoding
        else:
            real_encoding = self.get_charset(default="utf-8")
        try:
            decoded_data = data.rstrip().decode(real_encoding)
        except UnicodeDecodeError:
            raise ValueError("data cannot be decoded with %s encoding" % real_encoding)

        return parse_qsl(
            decoded_data,
            keep_blank_values=True,
            encoding=real_encoding,
        )

    def at_eof(self) -> bool:
        """Returns True if the boundary was reached or False otherwise."""
        return self._at_eof

    def _apply_content_transfer_decoding(self, data: bytes) -> bytes:
        """Apply Content-Transfer-Encoding decoding if header is present."""
        if CONTENT_TRANSFER_ENCODING in self.headers:
            return self._decode_content_transfer(data)
        return data

    def _needs_content_decoding(self) -> bool:

View on GitHub (pinned to c0ef574e29)

Solutions

1. Pass the correct charset explicitly: `await part.form(encoding='cp1252')` matching what the sender actually used.
2. Fix the sender to advertise the correct charset in the part's Content-Type header.
3. If the part is genuinely binary, do not use form() — read raw bytes via `await part.read(decode=True)` and handle them directly.
4. Wrap the call in try/except ValueError and surface a 400 to the client rather than crashing the handler.

Example fix

// before
fields = await part.form()
// after
try:
    fields = await part.form(encoding='cp1252')
except ValueError:
    fields = await part.read(decode=True)  # fall back to raw bytes

declared = part.get_charset(default='utf-8')
sample = await part.read_chunk(256)
try:
    sample.decode(declared)
    valid = True
except UnicodeDecodeError:
    valid = False
# rewind not possible on a stream — prefer passing explicit encoding to form()

def has_valid_charset(part) -> bool:
    enc = part.get_charset(default='utf-8')
    try:
        codecs.lookup(enc)
        return True
    except LookupError:
        return False

try:
    fields = await part.form(encoding='utf-8')
except ValueError as e:
    # charset mismatch — try fallback or reject
    fields = []

When it happens

Trigger: Calling `await part.form()` (or `await part.form(encoding='latin-1')`) on a BodyPartReader whose declared charset does not match the actual byte content. Typically happens when the upstream sender mislabels the charset (e.g. declares utf-8 but sends latin-1 / cp1252 bytes), or when binary data is accidentally routed through a text form part.

Common situations: Browser/upstream sending form fields with a wrong Content-Type charset; legacy servers emitting cp1252 labeled as utf-8; proxied requests where an intermediary re-encodes the body; debugging against curl with `--data-binary` that injects raw bytes.

Related errors

• Invalid default charset
• Cannot compute fallback encoding of a not yet read body
• Multipart field missing name.
• To decode nested multipart you need to use custom reader
• 1007

AI-assisted analysis of aio-libs/aiohttp@c0ef574e29 (2026-08-04). Data as JSON: /data/errors/055b980742b114c8.json. Report an issue: GitHub.