{"record":{"id":"032b8fe7892b51d6","repo":"unclecode/crawl4ai","slug":"nstproxy-invalid-api-response-expected-a-non-e","errorCode":null,"errorMessage":"[NSTProxy] Invalid API response — expected a non-empty list","messagePattern":"\\[NSTProxy\\] Invalid API response — expected a non-empty list","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crawl4ai/async_configs.py","lineNumber":1095,"sourceCode":"        if state:\n            params[\"state\"] = state\n        if city:\n            params[\"city\"] = city\n\n        url = \"https://api.nstproxy.com/api/v1/generate/apiproxies\"\n\n        try:\n            response = requests.get(url, params=params, timeout=10)\n            response.raise_for_status()\n\n            data = response.json()\n\n            # --- Handle API error response ---\n            if isinstance(data, dict) and data.get(\"err\"):\n                raise PermissionError(f\"[NSTProxy] API Error: {data.get('msg', 'Unknown error')}\")\n\n            if not isinstance(data, list) or not data:\n                raise ValueError(\"[NSTProxy] Invalid API response — expected a non-empty list\")\n\n            proxy_info = data[0]\n\n            # --- Apply proxy config ---\n            self.proxy_config = ProxyConfig(\n                server=f\"{protocol}://{proxy_info['ip']}:{proxy_info['port']}\",\n                username=proxy_info[\"username\"],\n                password=proxy_info[\"password\"],\n            )\n\n        except Exception as e:\n            print(f\"[NSTProxy] ❌ Failed to set proxy: {e}\")\n            raise\n\nclass VirtualScrollConfig:\n    \"\"\"Configuration for virtual scroll handling.\n    \n    This config enables capturing content from pages with virtualized scrolling","sourceCodeStart":1077,"sourceCodeEnd":1113,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/async_configs.py#L1077-L1113","documentation":"Raised when the NSTProxy API returns a 200 response whose JSON body is not a non-empty list. The code expects data = [{ip, port, username, password}, ...] and takes data[0]; anything else (empty list, dict without err, scalar) is treated as a contract violation.","triggerScenarios":"NSTProxy changing its response schema; an intermediary (corporate proxy, captive portal) returning JSON that is not the proxy list; the API returning [] when no proxy is currently available for the requested filters.","commonSituations":"Upstream API version bump; requesting a very narrow geo filter (country+state+city) with no inventory; TLS-inspecting middleboxes rewriting responses.","solutions":["Loosen the geo filters (drop state/city) and retry — empty availability is the most common cause","Manually curl https://api.nstproxy.com/api/v1/generate/apiproxies with your params to inspect the raw body and confirm the schema","If the schema changed, pin/upgrade the crawl4ai version that matches the current NSTProxy contract","Retry after a short delay — transient zero-inventory responses usually resolve"],"exampleFix":"// before\ncfg.set_nstproxy(token=t, channel_id=c, country=\"us\", state=\"ca\", city=\"tiny-town\")\n// after\ncfg.set_nstproxy(token=t, channel_id=c, country=\"us\")  # widen availability","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    cfg.set_nstproxy(token=t, channel_id=c)\nexcept ValueError as e:\n    if \"Invalid API response\" in str(e):\n        cfg.set_nstproxy(token=t, channel_id=c, country=\"\", state=\"\", city=\"\")  # widen filters\n    else:\n        raise","preventionTips":["Keep geo filters optional and fall back to broader ones","Log the raw API body when debugging schema issues","Pin a crawl4ai version tested against the NSTProxy API"],"tags":["proxy","api","schema","nstproxy","retry"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}