D4Vinci/Scrapling · error · ValueError

`autothrottle_max_delay` can't be lower than `autothrottle_s

Error message

`autothrottle_max_delay` can't be lower than `autothrottle_start_delay`

What it means

AutoThrottle requires max_delay >= start_delay; the throttle ramps delay from start_delay up to max_delay, so an inverted range is invalid and construction raises ValueError. Both values come from the spider's autothrottle_start_delay / autothrottle_max_delay settings.

Source

Thrown at scrapling/spiders/throttle.py:53

    def __init__(
        self,
        start_delay: float = 5.0,
        max_delay: float = 60.0,
        target_concurrency: float = 1.0,
        block_backoff: bool = True,
    ):
        """
        :param start_delay: The delay used for the first request to a domain.
        :param max_delay: The highest delay the throttle is allowed to reach.
        :param target_concurrency: How many requests the spider aims to have in flight per domain. The engine
            passes the spider's `concurrent_requests_per_domain` here, or 1 when it's unlimited.
        :param block_backoff: Double the delay of a domain whenever it blocks us, or wait what its `Retry-After`
            header asks for.
        """
        if target_concurrency <= 0:
            raise ValueError("`target_concurrency` must be higher than 0")
        if max_delay < start_delay:
            raise ValueError("`autothrottle_max_delay` can't be lower than `autothrottle_start_delay`")

        self.start_delay = start_delay
        self.max_delay = max_delay
        self.target_concurrency = target_concurrency
        self.block_backoff = block_backoff
        self.delays: Dict[str, float] = {}

    def delay_for(self, domain: str, floor: float = 0.0) -> float:
        """Return the current delay for a domain, starting it at `start_delay` the first time.

        :param domain: The domain the request belongs to.
        :param floor: The lowest delay allowed, which is the spider's own delay for this domain.
        """
        if domain not in self.delays:
            self.delays[domain] = min(max(floor, self.start_delay), self.max_delay)
        return self.delays[domain]

    def record(

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Ensure autothrottle_max_delay >= autothrottle_start_delay (e.g. start 1.0, max 10.0)
2. If you want a constant delay, set both to the same value
3. Re-check both fields after tuning either one

Example fix

// before
class MySpider(Spider):
    autothrottle_start_delay = 5.0
    autothrottle_max_delay = 2.0

// after
class MySpider(Spider):
    autothrottle_start_delay = 1.0
    autothrottle_max_delay = 10.0

assert autothrottle_max_delay >= autothrottle_start_delay, "max_delay must be >= start_delay"

When it happens

Trigger: Setting autothrottle_max_delay below autothrottle_start_delay on the spider class (e.g. start 5.0, max 2.0); copying delay presets where the two fields got swapped; tuning max_delay down without checking the start value.

Common situations: Aiming for faster crawls by lowering max_delay while the start_delay default is higher; presets/configs written for older defaults; unit tests constructing AutoThrottle with arbitrary numbers.

Related errors

• `target_concurrency` must be higher than 0
• Cannot use 'proxy_rotator' together with 'proxy' or 'proxies
• Unknown parser argument: "{key}"; maybe you meant {cls.parse
• You must pass a keyword to configure, current keywords: {cls
• Invalid proxy string!

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/4859c31b86f64087. Report an issue: GitHub.