D4Vinci/Scrapling · error · TypeError

Can't convert {type(value).__name__} to bytes

Error message

Can't convert {type(value).__name__} to bytes

What it means

The internal helper _convert_to_bytes accepts only str and bytes. Request bodies/headers constructed with any other type (int, float, dict, None) hit this TypeError, which names the offending type. Note that dicts/JSON are not auto-serialized — you must encode them yourself.

Source

Thrown at scrapling/spiders/request.py:20

from io import BytesIO
from functools import cached_property
from urllib.parse import urlparse, urlencode

import orjson
from w3lib.url import canonicalize_url

from scrapling.engines.toolbelt.custom import Response
from scrapling.core._types import Any, AsyncGenerator, Callable, Dict, Optional, Union, Tuple, TYPE_CHECKING

if TYPE_CHECKING:
    from scrapling.spiders.spider import Spider


def _convert_to_bytes(value: str | bytes) -> bytes:
    if isinstance(value, bytes):
        return value
    if not isinstance(value, str):
        raise TypeError(f"Can't convert {type(value).__name__} to bytes")

    return value.encode(encoding="utf-8", errors="ignore")


def _stable_value_repr(value: Any) -> str:
    try:
        return orjson.dumps(value, option=orjson.OPT_SORT_KEYS, default=repr).decode()
    except TypeError:
        return repr(value)


class Request:
    def __init__(
        self,
        url: str,
        sid: str = "",
        callback: Callable[[Response], AsyncGenerator[Union[Dict[str, Any], "Request", None], None]] | None = None,
        priority: int = 0,

View on GitHub (pinned to 5d213a2d47)

Solutions

1. JSON-encode dict/list bodies first: body=orjson.dumps(payload) or json.dumps(payload).encode().
2. Convert numerics: str(value) for headers, str(value).encode() for bodies.
3. Guard against None: body = body if body is not None else b''.

Example fix

# before
Request(url, body={'q': 'x'})  # TypeError: Can't convert dict to bytes

# after
import json
Request(url, body=json.dumps({'q': 'x'}).encode('utf-8'))

def as_body(value) -> bytes:
    if value is None:
        return b''
    if isinstance(value, bytes):
        return value
    if isinstance(value, str):
        return value.encode('utf-8')
    if isinstance(value, (dict, list)):
        return json.dumps(value).encode('utf-8')
    raise TypeError(f'unsupported body type {type(value).__name__}')

Request(url, body=as_body(payload))

def is_str_or_bytes(v) -> bool:
    return isinstance(v, (str, bytes))

When it happens

Trigger: Request(url, body={'q': 'x'}), Request(url, data=123), or a header value computed as an int (e.g. content_length=size) rather than str(size).

Common situations: Passing a parsed JSON payload straight back as a body; building headers from numeric variables; a None default leaking into a body argument.

Related errors

• Response meta should be dictionary but got {type(meta).__nam
• Invalid proxy string: {proxy_string}
• strategy must be callable, got {type(strategy).__name__}
• Invalid proxy type: {type(proxy)}. Expected str or dict.
• Argument `selector_config` must be a dictionary.

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/53172d34f0cd248f. Report an issue: GitHub.