D4Vinci/Scrapling · error · ValueError

`target_concurrency` must be higher than 0

Error message

`target_concurrency` must be higher than 0

What it means

AutoThrottle validates that target_concurrency > 0 at construction. The engine passes the spider's concurrent_requests_per_domain here (or 1 when unlimited), so a zero or negative concurrency setting surfaces as this ValueError.

Source

Thrown at scrapling/spiders/throttle.py:51

    servers and backs off on slow or hostile ones."""

    def __init__(
        self,
        start_delay: float = 5.0,
        max_delay: float = 60.0,
        target_concurrency: float = 1.0,
        block_backoff: bool = True,
    ):
        """
        :param start_delay: The delay used for the first request to a domain.
        :param max_delay: The highest delay the throttle is allowed to reach.
        :param target_concurrency: How many requests the spider aims to have in flight per domain. The engine
            passes the spider's `concurrent_requests_per_domain` here, or 1 when it's unlimited.
        :param block_backoff: Double the delay of a domain whenever it blocks us, or wait what its `Retry-After`
            header asks for.
        """
        if target_concurrency <= 0:
            raise ValueError("`target_concurrency` must be higher than 0")
        if max_delay < start_delay:
            raise ValueError("`autothrottle_max_delay` can't be lower than `autothrottle_start_delay`")

        self.start_delay = start_delay
        self.max_delay = max_delay
        self.target_concurrency = target_concurrency
        self.block_backoff = block_backoff
        self.delays: Dict[str, float] = {}

    def delay_for(self, domain: str, floor: float = 0.0) -> float:
        """Return the current delay for a domain, starting it at `start_delay` the first time.

        :param domain: The domain the request belongs to.
        :param floor: The lowest delay allowed, which is the spider's own delay for this domain.
        """
        if domain not in self.delays:
            self.delays[domain] = min(max(floor, self.start_delay), self.max_delay)
        return self.delays[domain]

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Set concurrent_requests_per_domain to a positive value (e.g. 1 for polite, higher for aggressive crawling)
2. If unlimited per-domain concurrency is desired, leave the setting unset/None so the engine passes 1 to the throttle with unlimited in-flight requests
3. When building AutoThrottle yourself, pass max(1, concurrency)

Example fix

// before
class MySpider(Spider):
    concurrent_requests_per_domain = 0  # rejected

// after
class MySpider(Spider):
    concurrent_requests_per_domain = 4

concurrency = max(1, concurrency)  # before assigning to the spider/throttle

When it happens

Trigger: Setting concurrent_requests_per_domain = 0 (or a negative number) on the spider, which the engine forwards as target_concurrency to AutoThrottle; constructing AutoThrottle directly with target_concurrency=0.

Common situations: Using 0 to mean 'unlimited' — in this library unlimited is expressed differently and 0 is rejected; config files that default concurrency to 0; arithmetic that computes concurrency from data and can produce 0.

Related errors

• `autothrottle_max_delay` can't be lower than `autothrottle_s
• Cannot use 'proxy_rotator' together with 'proxy' or 'proxies
• Unknown parser argument: "{key}"; maybe you meant {cls.parse
• You must pass a keyword to configure, current keywords: {cls
• Invalid proxy string!

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/483efd9fedfcc1f7. Report an issue: GitHub.