{"record":{"id":"ec47b2bcf96e8eaf","repo":"assafelovic/gpt-researcher","slug":"error-parsing-searxng-response","errorCode":null,"errorMessage":"Error parsing SearxNG response","messagePattern":"Error parsing SearxNG response","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"gpt_researcher/retrievers/searx/searx.py","lineNumber":88,"sourceCode":"        params = {\n            # The search query.\n            'q': self.query,\n            # Output format of results. Format needs to be activated in searxng config.\n            'format': 'json'\n        }\n\n        try:\n            response = requests.get(\n                search_url,\n                params=params,\n                headers={'Accept': 'application/json'}\n            )\n            response.raise_for_status()\n            results = response.json()\n        except requests.exceptions.RequestException as e:\n            raise Exception(f\"Error querying SearxNG: {str(e)}\")\n        except json.JSONDecodeError:\n            raise Exception(\"Error parsing SearxNG response\")\n\n        if not isinstance(results, dict):\n            return []\n\n        search_response = []\n        raw_results = results.get('results', [])\n        if not isinstance(raw_results, list):\n            return []\n\n        for result in raw_results:\n            if not isinstance(result, dict):\n                continue\n            href = result.get('url') or result.get('href') or ''\n            if not href:\n                continue\n            body = result.get('content') or result.get('snippet') or ''\n            search_response.append({\n                \"href\": href,","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/assafelovic/gpt-researcher/blob/6f998577d547b1e54ec662dac63583aa11e3b84b/gpt_researcher/retrievers/searx/searx.py#L70-L106","documentation":"Exception raised by SearxNG search() when the instance returns HTTP 200 but the body is not valid JSON (json.JSONDecodeError caught and re-raised as this plain message). Most commonly the instance returned an HTML page because JSON output is disabled for API clients.","triggerScenarios":"search() calls response.json(); the SearxNG instance responds with HTML (error page, CAPTCHA, or default HTML format) instead of the requested JSON.","commonSituations":"Self-hosted SearxNG without 'json' in search.formats, public instances that disable the JSON API to prevent scraping, or a reverse proxy/WAF serving an HTML error page.","solutions":["Enable JSON output on your instance: in settings.yml set search: formats: [html, json] and restart.","If using a public instance, pick one from searx.space that allows format=json, or self-host.","Check response.text of the URL directly to see what HTML is being served (CAPTCHA/proxy error).","Ensure no proxy is rewriting the Accept: application/json request."],"exampleFix":"# searx settings.yml — before\nsearch:\n  formats:\n    - html\n\n# after\nsearch:\n  formats:\n    - html\n    - json\n# then restart searxng","handlingStrategy":"validation","validationCode":"import requests, os\nurl = os.environ[\"SEARX_URL\"].rstrip('/') + '/search'\nr = requests.get(url, params={'q': 'test', 'format': 'json'}, timeout=5,\n                 headers={'Accept': 'application/json'})\nassert r.headers.get('content-type', '').startswith('application/json'), \\\n    \"SearxNG JSON output disabled — enable 'json' in search.formats\"","typeGuard":"null","tryCatchPattern":"try:\n    results = retriever.search()\nexcept Exception as e:\n    if \"Error parsing\" in str(e):\n        logger.error(\"SearxNG returned HTML; enable format=json on the instance\")\n        results = []\n    else:\n        raise","preventionTips":["Enable 'json' in the instance's settings.yml search.formats.","Avoid public instances that block the JSON API; self-host instead.","Check content-type of the instance's response during setup."],"tags":["searxng","json-decode","configuration","retriever"],"backgroundTag":"invalid-json-response","analyzedSha":"6f998577d547b1e54ec662dac63583aa11e3b84b","analyzedAt":"2026-08-28T17:50:07.383Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}