{"record":{"id":"278aaa901984ff80","repo":"zylon-ai/private-gpt","slug":"web-fetching-is-not-properly-initialized-or-it-is-278aaa","errorCode":null,"errorMessage":"Web fetching is not properly initialized or it is disabled. Since Web Search depends on web fetching to retrieve content, the Web Search functionality cannot operate correctly. Consider enabling web fetching in settings.","messagePattern":"Web fetching is not properly initialized or it is disabled\\. Since Web Search depends on web fetching to retrieve content, the Web Search functionality cannot operate correctly\\. Consider enabling web fetching in settings\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/web/web_search/processors/clean_content.py","lineNumber":34,"sourceCode":"    )\n\n\nlogger = logging.getLogger(__name__)\n\n\nclass CleanContentProcessor(BaseWebSearchResultProcessor):\n    def __init__(\n        self,\n        settings: Settings,\n        scraper_service: WebScraperService,\n    ) -> None:\n        super().__init__()\n        self._settings = settings\n        self._scraper_service = scraper_service\n\n    async def validate(self) -> None:\n        if not self._scraper_service.is_initialized:\n            raise ValueError(\n                \"Web fetching is not properly initialized or it is disabled. \"\n                \"Since Web Search depends on web fetching to retrieve content, \"\n                \"the Web Search functionality cannot operate correctly. \"\n                \"Consider enabling web fetching in settings.\"\n            )\n\n    async def process_results(\n        self,\n        query: str,\n        results: list[WebSearchResult],\n        model_id: str | None = None,\n    ) -> list[WebSearchResult]:\n        processed_results: list[WebSearchResult] = []\n        for result in results[0 : self._settings.web_search.num_links]:\n            try:\n                # Get HTML for the result - lazy load scraper service\n                response: WebScraperResult = (\n                    await self._scraper_service.scrape_max_compress(result.url)","sourceCodeStart":16,"sourceCodeEnd":52,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/web/web_search/processors/clean_content.py#L16-L52","documentation":"ValueError from CleanContentProcessor.validate(): the web search pipeline was asked to fetch and clean result pages, but the shared WebScraperService reports is_initialized == False, meaning settings.web_fetch.enabled was false at startup. Web search's clean_content processor depends on web fetching to retrieve page bodies, so validation aborts before any query runs.","triggerScenarios":"web_search.enabled: true with processor: clean_content but web_fetch.enabled left false (its default); calling WebSearchService.search() which first awaits validate() → processor.validate().","commonSituations":"Enabling web search without realizing its content processors require the separate web-fetch flag; config split across profiles where the fetch toggle lives in a profile that is not active.","solutions":["Enable web fetching: web_fetch.enabled: true alongside web_search.enabled: true.","If you do not want page fetching, switch web_search.processor to simple_text which has no scraper dependency.","Restart the service after settings changes — initialization flags are set once at startup."],"exampleFix":"# settings.yaml\nweb_search:\n  enabled: true\n  processor: clean_content\nweb_fetch:\n  enabled: true  # required by clean_content","handlingStrategy":"validation","validationCode":"async def web_search_ready(search_svc: WebSearchService,\n                              scraper_svc: WebScraperService) -> bool:\n    return (\n        search_svc._initialized\n        and scraper_svc.is_initialized  # processor's requirement\n    )","typeGuard":null,"tryCatchPattern":"try:\n    results = await search_svc.search(query)\nexcept ValueError as e:\n    if 'Web Search depends on web fetching' in str(e):\n        raise RuntimeError('Config error: enable web_fetch for clean_content processor') from e\n    raise","preventionTips":["Choose simple_text processor when web fetching is intentionally disabled.","Run validate() in a health endpoint so misconfig is visible before user traffic.","Encapsulate 'web search requires web fetch' in deployment docs and config examples."],"tags":["configuration","web-search","web-fetch","validation"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}