NanmiCoder/MediaCrawler · error · Exception

get ip error from proxy provider and status code not 200 ...

Error message

get ip error from proxy provider and status code not 200 ...

What it means

Raised in KuaiDaiLiProxy.get_proxies() when the HTTP request to the KuaiDaiLi API (api_base + uri with self.params) returns a non-200 status code. The full response body is logged beforehand via utils.logger.error, so the provider's own error text is available in logs. This is a transport/auth-level failure, distinct from a 200 response whose JSON body carries a non-zero code.

Source

Thrown at proxy/providers/kuaidl_proxy.py:122

        """
        uri = "/api/getdps/"

        # Prioritize getting IP from cache
        ip_cache_list = self.ip_cache.load_all_ip(proxy_brand_name=self.proxy_brand_name)
        if len(ip_cache_list) >= num:
            return ip_cache_list[:num]

        # If the quantity in cache is insufficient, get from IP provider to supplement, then store in cache
        need_get_count = num - len(ip_cache_list)
        self.params.update({"num": need_get_count})

        ip_infos: List[IpInfoModel] = []
        async with make_async_client() as client:
            response = await client.get(self.api_base + uri, params=self.params)

            if response.status_code != 200:
                utils.logger.error(f"[KuaiDaiLiProxy.get_proxies] statuc code not 200 and response.txt:{response.text}, status code: {response.status_code}")
                raise Exception("get ip error from proxy provider and status code not 200 ...")

            ip_response: Dict = response.json()
            if ip_response.get("code") != 0:
                utils.logger.error(f"[KuaiDaiLiProxy.get_proxies]  code not 0 and msg:{ip_response.get('msg')}")
                raise Exception("get ip error from proxy provider and  code not 0 ...")

            proxy_list: List[str] = ip_response.get("data", {}).get("proxy_list")
            for proxy in proxy_list:
                proxy_model = parse_kuaidaili_proxy(proxy)
                # expire_ts is relative time (seconds), needs to be converted to absolute timestamp
                # Consider expired DELTA_EXPIRED_SECOND seconds in advance to avoid critical time usage failure
                ip_info_model = IpInfoModel(
                    ip=proxy_model.ip,
                    port=proxy_model.port,
                    user=self.kdl_user_name,
                    password=self.kdl_user_pwd,
                    expired_time_ts=proxy_model.expire_ts + utils.get_unix_timestamp() - DELTA_EXPIRED_SECOND,

View on GitHub (pinned to d6f7c5bb90)

Solutions

  1. Read the logged response.text — KuaiDaili usually states the exact reason (auth failure, IP not whitelisted, rate limit)
  2. Add the outbound server IP to the KuaiDaili account's IP whitelist and retry
  3. Verify KDL_SECRET_ID / KDL_SIGNATURE env vars match the current KuaiDaili order credentials
  4. Throttle get_proxies() calls or raise the interval to stay under the order's extraction frequency
  5. Wrap the call in a retry with backoff for transient 5xx/429 responses

Example fix

// before
response = await client.get(self.api_base + uri, params=self.params)
if response.status_code != 200:
    raise Exception("get ip error from proxy provider and status code not 200 ...")

// after
response = await client.get(self.api_base + uri, params=self.params)
if response.status_code in (429, 502, 503):
    await asyncio.sleep(2)
    response = await client.get(self.api_base + uri, params=self.params)
if response.status_code != 200:
    raise IpGetError(f"kdl http {response.status_code}: {response.text[:200]}")
Defensive patterns

Strategy: retry

Validate before calling

async def kdl_api_reachable(client) -> bool:
    try:
        r = await client.get(api_base + health_uri, timeout=5)
        return r.status_code < 500
    except Exception:
        return False

Try / catch

for attempt in range(3):
    try:
        return await kdl.get_proxies(num=n)
    except Exception as e:  # status!=200 path
        if attempt == 2:
            raise
        utils.logger.warning(f"kdl http error, backoff retry {attempt+1}: {e}")
        await asyncio.sleep(2 ** attempt)

Prevention

When it happens

Trigger: GET to the KuaiDaiLi /GetKdlIp endpoint with invalid or expired signature/secret params (401/403), frequency-limit breach (429 or KuaiDaili's custom status), missing whitelist of your server IP in the KuaiDaiLi console, or network egress blocked so an intermediary returns 4xx/5xx.

Common situations: KuaiDaiLi account credentials (KDL_SECRET_ID / KDL_SIGNATURE) rotated or expired; the crawler server IP not added to the provider's IP whitelist; running many concurrent crawlers that exceed the extraction QPS; corporate firewall/proxy mangling the request.

Related errors

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/7d36d5077290d91f. Report an issue: GitHub.