{"record":{"id":"d3dd4b2084024f95","repo":"oobabooga/textgen","slug":"too-many-redirects-max-max-redirects","errorCode":null,"errorMessage":"Too many redirects (max {max_redirects})","messagePattern":"Too many redirects \\(max (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"modules/web_search.py","lineNumber":54,"sourceCode":"            ip = ipaddress.ip_address(sockaddr[0])\n            if not ip.is_global:\n                raise ValueError(f\"Access to non-public address {ip} is blocked\")\n    except socket.gaierror:\n        raise ValueError(f\"Could not resolve hostname: {hostname}\")\n\n\ndef safe_get(url, headers=None, timeout=10, max_redirects=5):\n    \"\"\"Fetch a URL with SSRF-safe redirect handling. Validates every hop.\"\"\"\n    _validate_url(url)\n    for _ in range(max_redirects):\n        response = requests.get(url, headers=headers, timeout=timeout, allow_redirects=False)\n        if response.is_redirect and 'Location' in response.headers:\n            url = urljoin(url, response.headers['Location'])\n            _validate_url(url)\n        else:\n            return response\n\n    raise ValueError(f\"Too many redirects (max {max_redirects})\")\n\n\ndef get_current_timestamp():\n    \"\"\"Returns the current time in 24-hour format\"\"\"\n    return datetime.now().strftime('%b %d, %Y %H:%M')\n\n\ndef download_web_page(url, timeout=10, include_links=False):\n    \"\"\"\n    Download a web page and extract its main content as Markdown text.\n    \"\"\"\n    import trafilatura\n\n    try:\n        headers = {\n            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36'\n        }\n        response = safe_get(url, headers=headers, timeout=timeout)","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/oobabooga/textgen/blob/ed888c71f221df552750e1834b3654abab8ae345/modules/web_search.py#L36-L72","documentation":"Raised by safe_get() when the redirect chain exceeds max_redirects (default 5) hops without a terminal response. The function follows redirects manually (allow_redirects=False) so that every hop can be re-validated by _validate_url(); once the budget is exhausted it refuses to continue rather than looping indefinitely, protecting against redirect loops and chains used to exhaust resources.","triggerScenarios":"Fetching a URL involved in a redirect loop (A -> B -> A), an excessively long chain (shortlink services chaining through many trackers), or a server that keeps issuing redirects (e.g. HTTP->HTTPS->auth->back) exceeding 5 hops. Each hop also costs a validation pass, so the cap bounds total work.","commonSituations":"Login-walled or CDN-protected sites that redirect repeatedly; broken server configs with self-redirects; tracker-laden short links; mirroring/crawling tools that hit sites with cookie-consent redirect churn.","solutions":["Fetch the final URL directly: open the link once in a browser/curl -L, take the landed URL, and pass that to safe_get.","Retry with a higher max_redirects if you control the call and trust the target (safe_get accepts max_redirects as a parameter).","If the site is redirect-looping due to missing cookies/headers, pass appropriate headers (User-Agent, cookies) to break the loop.","Skip the offending link in bulk crawling scenarios and report it as unreachable."],"exampleFix":"# before\nresp = safe_get(url)  # default max_redirects=5, loop site raises\n\n# after\nresp = safe_get(url, max_redirects=10)  # or first resolve the final URL out-of-band\n# final = requests.head(url, allow_redirects=True).url\n# resp = safe_get(final)","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    resp = safe_get(url)\nexcept ValueError as e:\n    if 'Too many redirects' in str(e):\n        # resolve final URL out-of-band once, then fetch directly\n        import requests\n        final = requests.head(url, allow_redirects=True, timeout=10).url\n        resp = safe_get(final)\n    else:\n        raise","preventionTips":["Resolve short links to their final URL once and cache it, rather than re-following chains each fetch.","Pass a realistic User-Agent; many consent/CDN redirect loops key off default client strings.","Set an explicit max_redirects budget in crawlers and skip links that exhaust it."],"tags":["web-fetch","redirects","network","ssrf"],"backgroundTag":null,"analyzedSha":"ed888c71f221df552750e1834b3654abab8ae345","analyzedAt":"2026-08-15T05:24:21.000Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}