D4Vinci/Scrapling · error · RuntimeError

No active crawl. Use this property inside `async for item in

Error message

No active crawl. Use this property inside `async for item in spider.stream():`

What it means

The Spider.stats property returns the live CrawlStats only while self._engine exists (i.e., during an active crawl). Outside `async for item in spider.stream()` — before starting, after the loop ends, or after start() returned — the engine is set to None and the property raises RuntimeError.

Source

Thrown at scrapling/spiders/spider.py:330

        token = set_logger(self.logger)
        try:
            self._engine = CrawlerEngine(self, self._session_manager, self.crawldir, self._interval)
            async for item in self._engine:
                yield item
        finally:
            self._engine = None
            reset_logger(token)
            if self.log_file:
                for handler in self.logger.handlers:
                    if isinstance(handler, logging.FileHandler):
                        handler.close()

    @property
    def stats(self) -> CrawlStats:
        """Access current crawl stats (works during streaming)."""
        if self._engine:
            return self._engine.stats
        raise RuntimeError("No active crawl. Use this property inside `async for item in spider.stream():`")

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Inside the stream loop, read spider.stats while iterating (e.g. every N items)
2. For final stats with start(), capture the returned CrawlResult: result = spider.start(); result.stats
3. Move any post-crawl stats reporting to use the CrawlResult/CrawlStats object instead of the spider property

Example fix

// before
async for item in spider.stream():
    ...
print(spider.stats)  # engine already cleared -> RuntimeError

// after
result = spider.start()
print(result.stats)

stats = spider.stats if spider._engine else None  # read only during the stream loop

def crawl_active(spider) -> bool:
    return spider._engine is not None

try:
    stats = spider.stats
except RuntimeError:
    stats = None  # crawl not running

When it happens

Trigger: Accessing spider.stats before calling start()/stream(); reading it after the stream generator is exhausted or the loop breaks (the finally block in stream() sets _engine = None); logging stats in a finally/after clause outside the iteration.

Common situations: Trying to print final stats after the crawl loop — the correct result source is the CrawlResult returned by start(), not spider.stats; accessing stats in cleanup code after an exception ended the stream.

Related errors

• Session '{session_id}' is no longer alive. Open a new sessio
• Browser not initialized for proxy rotation mode
• Session has been already started
• Context manager has been closed
• No active crawl to stop

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/7c0419dd44a53bdb. Report an issue: GitHub.