zylon-ai/private-gpt · error · ValueError

Web fetching is not properly initialized or it is disabled.

Error message

Web fetching is not properly initialized or it is disabled. Consider enabling web fetching in settings.

What it means

ValueError raised by WebScraperService._scrape_html when the singleton was never marked initialized. _initialize() skips initialization when settings.web_fetch.enabled is False (the default), leaving _initialized False; every later scrape call trips this guard.

Source

Thrown at private_gpt/components/web/web_scraper_service.py:80

    def is_initialized(self) -> bool:
        return self._initialized

    def _apply_ssl_environment(self) -> None:
        ssl_config = self._settings.server.network.ssl
        if ssl_config.cert_file:
            os.environ["NODE_EXTRA_CA_CERTS"] = ssl_config.cert_file
            os.environ["SSL_CERT_FILE"] = ssl_config.cert_file
            logger.debug(f"Custom CA certificate configured: {ssl_config.cert_file}")

        if ssl_config.cert_dir:
            os.environ["SSL_CERT_DIR"] = ssl_config.cert_dir
            logger.debug(
                f"Custom CA certificate directory configured: {ssl_config.cert_dir}"
            )

    async def _scrape_html(self, url: str) -> str:
        if not self._initialized:
            raise ValueError(
                "Web fetching is not properly initialized or it is disabled. "
                "Consider enabling web fetching in settings."
            )

        provider = self._registry.get_provider(self._settings.web_fetch.provider)
        _start_scrape = time.monotonic()
        try:
            content = await provider.scrape_html(
                url, self._settings.web_fetch.timeout_seconds
            )
        except TimeoutError:
            logger.warning(
                f"Timeout ({self._settings.web_fetch.timeout_seconds}s) scraping {url}"
            )
            raise
        except Exception as e:
            logger.warning(f"Error scraping {url}: {e}")
            raise

View on GitHub (pinned to 4a030776a3)

Solutions

Set web_fetch.enabled: true in settings (profile yaml or env var PRIVATE_GPT_WEB_FETCH_ENABLED=true).
If web search is the goal, enable both web_fetch and web_search.
Verify with the service's is_initialized property after startup to catch config drift early.

Example fix

# settings.yaml
web_fetch:
  enabled: true  # was false / missing
  provider: local

Defensive patterns

Strategy: validation

Validate before calling

svc = get_global_injector().get(WebScraperService)
if not svc.is_initialized:
    raise ValueError('web_fetch.enabled is false — enable it before scraping')

Type guard

def scraper_ready(svc: WebScraperService) -> bool:
    return svc.is_initialized

Try / catch

try:
    html = await svc.scrape(url)
except ValueError as e:
    if 'Web fetching is not properly initialized' in str(e):
        # config problem, not a page problem: surface to operator
        raise RuntimeError('Fix settings: web_fetch.enabled=true') from e
    raise

Prevention

Assert scraper.is_initialized during app startup, not at first scrape.
Keep web_fetch and web_search toggles together in config templates.
Add a config lint that warns when web_search.enabled=true but web_fetch.enabled=false.

When it happens

Trigger: Default settings (web_fetch.enabled defaults to false) combined with any code path that scrapes; enabling web search but not web fetch; settings profile/env var not loaded so the enabled flag stays default.

Common situations: Fresh deployment where web fetching was never turned on; users enabling `web_search.enabled` but forgetting `web_fetch.enabled`; YAML typo under a different profile so the override never applies.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/5b3695ca45679910. Report an issue: GitHub.