{"record":{"id":"1a58893bc5dc3084","repo":"zylon-ai/private-gpt","slug":"web-fetching-is-not-properly-initialized-or-it-is-1a5889","errorCode":null,"errorMessage":"Web fetching is not properly initialized or it is disabled. Since Web Search depends on web fetching to retrieve content, the Web Search functionality cannot operate correctly. Consider enabling web fetching in settings.","messagePattern":"Web fetching is not properly initialized or it is disabled\\. Since Web Search depends on web fetching to retrieve content, the Web Search functionality cannot operate correctly\\. Consider enabling web fetching in settings\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/web/web_search/processors/select_best_links.py","lineNumber":70,"sourceCode":"    _max_tokens: int = 0\n\n    def __init__(\n        self,\n        settings: Settings,\n        scraper_service: WebScraperService,\n        llm_component: LLMComponent,\n        summary_builder: SummarizeWorkflowBuilder,\n    ) -> None:\n        super().__init__()\n        self._settings = settings\n        self._scraper_service = scraper_service\n        self._llm_component = llm_component\n        self._summary_builder = summary_builder\n        self._init_params()\n\n    async def validate(self) -> None:\n        if not self._scraper_service.is_initialized:\n            raise ValueError(\n                \"Web fetching is not properly initialized or it is disabled. \"\n                \"Since Web Search depends on web fetching to retrieve content, \"\n                \"the Web Search functionality cannot operate correctly. \"\n                \"Consider enabling web fetching in settings.\"\n            )\n\n    async def process_results(\n        self,\n        query: str,\n        results: list[WebSearchResult],\n        model_id: str | None = None,\n    ) -> list[WebSearchResult]:\n        start = time.perf_counter()\n        max_tokens = int(\n            (\n                self._settings.web_search.context_token\n                or self._llm_component.metadata(model_id).context_window\n            )","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/web/web_search/processors/select_best_links.py#L52-L88","documentation":"ValueError from SelectBestLinks.validate(): the best_links processor ranks and summarizes result pages, which requires actually fetching them, so it validates that the shared WebScraperService was initialized. is_initialized is False because web_fetch.enabled defaulted to false, so web search with processor best_links refuses to run.","triggerScenarios":"web_search.enabled: true, web_search.processor: best_links, but web_fetch.enabled false; SelectBestLinks also requires llm_component and summary_builder, but this specific error is the scraper gate in validate().","commonSituations":"Copying a web-search config block without the matching web-fetch block; disabling web fetch later for cost while keeping the best_links processor.","solutions":["Set web_fetch.enabled: true in settings.","Or pick a processor without scraper dependency (simple_text).","Verify both flags in the active profile, then restart so singletons re-initialize."],"exampleFix":"# settings.yaml\nweb_search:\n  enabled: true\n  processor: best_links\nweb_fetch:\n  enabled: true  # required by best_links","handlingStrategy":"validation","validationCode":"if settings.web_search.processor == 'best_links' and not settings.web_fetch.enabled:\n    raise ValueError('best_links processor requires web_fetch.enabled=true')","typeGuard":null,"tryCatchPattern":"try:\n    results = await search_svc.search(query)\nexcept ValueError as e:\n    if 'Web Search depends on web fetching' in str(e):\n        raise RuntimeError('Enable web_fetch for best_links processor') from e\n    raise","preventionTips":["Validate the processor/web_fetch combination in a config sanity check at boot.","Remember best_links also needs llm_component + summary_builder — test with validate().","Keep a matrix of processor → required settings in docs."],"tags":["configuration","web-search","web-fetch","validation"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}