zylon-ai/private-gpt · error · ValueError

Web fetching is not properly initialized or it is disabled.

Error message

Web fetching is not properly initialized or it is disabled. Since Web Search depends on web fetching to retrieve content, the Web Search functionality cannot operate correctly. Consider enabling web fetching in settings.

What it means

ValueError from SelectBestLinks.validate(): the best_links processor ranks and summarizes result pages, which requires actually fetching them, so it validates that the shared WebScraperService was initialized. is_initialized is False because web_fetch.enabled defaulted to false, so web search with processor best_links refuses to run.

Source

Thrown at private_gpt/components/web/web_search/processors/select_best_links.py:70

    _max_tokens: int = 0

    def __init__(
        self,
        settings: Settings,
        scraper_service: WebScraperService,
        llm_component: LLMComponent,
        summary_builder: SummarizeWorkflowBuilder,
    ) -> None:
        super().__init__()
        self._settings = settings
        self._scraper_service = scraper_service
        self._llm_component = llm_component
        self._summary_builder = summary_builder
        self._init_params()

    async def validate(self) -> None:
        if not self._scraper_service.is_initialized:
            raise ValueError(
                "Web fetching is not properly initialized or it is disabled. "
                "Since Web Search depends on web fetching to retrieve content, "
                "the Web Search functionality cannot operate correctly. "
                "Consider enabling web fetching in settings."
            )

    async def process_results(
        self,
        query: str,
        results: list[WebSearchResult],
        model_id: str | None = None,
    ) -> list[WebSearchResult]:
        start = time.perf_counter()
        max_tokens = int(
            (
                self._settings.web_search.context_token
                or self._llm_component.metadata(model_id).context_window
            )

View on GitHub (pinned to 4a030776a3)

Solutions

Set web_fetch.enabled: true in settings.
Or pick a processor without scraper dependency (simple_text).
Verify both flags in the active profile, then restart so singletons re-initialize.

Example fix

# settings.yaml
web_search:
  enabled: true
  processor: best_links
web_fetch:
  enabled: true  # required by best_links

Defensive patterns

Strategy: validation

Validate before calling

if settings.web_search.processor == 'best_links' and not settings.web_fetch.enabled:
    raise ValueError('best_links processor requires web_fetch.enabled=true')

Try / catch

try:
    results = await search_svc.search(query)
except ValueError as e:
    if 'Web Search depends on web fetching' in str(e):
        raise RuntimeError('Enable web_fetch for best_links processor') from e
    raise

Prevention

Validate the processor/web_fetch combination in a config sanity check at boot.
Remember best_links also needs llm_component + summary_builder — test with validate().
Keep a matrix of processor → required settings in docs.

When it happens

Trigger: web_search.enabled: true, web_search.processor: best_links, but web_fetch.enabled false; SelectBestLinks also requires llm_component and summary_builder, but this specific error is the scraper gate in validate().

Common situations: Copying a web-search config block without the matching web-fetch block; disabling web fetch later for cost while keeping the best_links processor.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/1a58893bc5dc3084. Report an issue: GitHub.