zylon-ai/private-gpt · error · ValueError

Web scraper provider '{name}' is not registered. Available:

Error message

Web scraper provider '{name}' is not registered. Available: {available}

What it means

ValueError raised by the web scraper registry's get_provider when the requested provider name is neither cached in the instance map nor present in the module-level _PROVIDERS factory map. The message enumerates the registered provider names.

Source

Thrown at private_gpt/components/web/scraper/registry.py:30

    name: str, provider: WebScraperProviderFactory
) -> None:
    _PROVIDERS[name] = provider


class WebScraperProviderRegistry:
    def __init__(self, settings: Settings) -> None:
        self._settings = settings
        self._providers: dict[str, WebScraperProvider] = {}

    def get_provider(self, name: str) -> WebScraperProvider:
        provider = self._providers.get(name)
        if provider is not None:
            return provider

        provider_factory = _PROVIDERS.get(name)
        if provider_factory is None:
            available = ", ".join(sorted(_PROVIDERS)) or "none"
            raise ValueError(
                f"Web scraper provider '{name}' is not registered. "
                f"Available: {available}"
            )
        provider = provider_factory(self._settings)
        self._providers[name] = provider
        return provider

    async def close_all(self) -> None:
        providers, self._providers = list(self._providers.values()), {}
        for provider in providers:
            await provider.close()

View on GitHub (pinned to 4a030776a3)

Solutions

Use one of the provider names listed in the error message (exact spelling/case)
For custom scrapers, ensure their module is imported and their factory is added to _PROVIDERS before get_provider is called
Log sorted(_PROVIDERS) at startup to keep an inventory of valid names

Example fix

# before
provider = registry.get_provider("BeautifulSoup")
# after
provider = registry.get_provider("beautifulsoup")  # exact key from _PROVIDERS

Defensive patterns

Strategy: validation

Validate before calling

from private_gpt.components.web.scraper.registry import _PROVIDERS

def provider_exists(name: str) -> bool:
    return name in _PROVIDERS

assert provider_exists(name), f"unknown web scraper {name!r}; known: {sorted(_PROVIDERS)}"

Try / catch

try:
    provider = registry.get_provider(name)
except ValueError as e:
    if "is not registered" in str(e):
        # parse the Available list from the message, correct the name, retry once
        raise

Prevention

Drive scraper selection from an enum/constant list, not free-form config strings
Import custom scraper modules at startup so their _PROVIDERS entries exist before use
Log sorted(_PROVIDERS) at boot to keep an accurate inventory

When it happens

Trigger: Calling get_provider(name) with a misspelled or unregistered provider (e.g. 'trafilatura' vs 'trafilatura_scraper', or a plugin provider whose registration never ran); case-sensitive mismatch.

Common situations: Settings-driven scraper selection (web.scraper.provider) with a stale name after an upgrade; custom scrapers expected to auto-register but their module was never imported; copy-pasted provider names from other projects.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/507121ab03149e51. Report an issue: GitHub.