{"record":{"id":"063b32cdaac6d40c","repo":"microsoft/qlib","slug":"response-status-status-url-url","errorCode":null,"errorMessage":"response status: {_status}, url={url}","messagePattern":"response status: (.+?), url=(.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/data_collector/cn_index/collector.py","lineNumber":45,"sourceCode":")\n\n\nINDEX_CHANGES_URL = \"https://www.csindex.com.cn/csindex-home/search/search-content?lang=cn&searchInput=%E5%85%B3%E4%BA%8E%E8%B0%83%E6%95%B4%E6%B2%AA%E6%B7%B1300%E5%92%8C%E4%B8%AD%E8%AF%81%E9%A6%99%E6%B8%AF100%E7%AD%89%E6%8C%87%E6%95%B0%E6%A0%B7%E6%9C%AC&pageNum={page_num}&pageSize={page_size}&sortField=date&dateRange=all&contentType=announcement\"\n\nREQ_HEADERS = {\n    \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36 Edg/91.0.864.48\"\n}\n\n\n@deco_retry\ndef retry_request(url: str, method: str = \"get\", exclude_status: List = None):\n    if exclude_status is None:\n        exclude_status = []\n    method_func = getattr(requests, method)\n    _resp = method_func(url, headers=REQ_HEADERS, timeout=None)\n    _status = _resp.status_code\n    if _status not in exclude_status and _status != 200:\n        raise ValueError(f\"response status: {_status}, url={url}\")\n    return _resp\n\n\nclass CSIIndex(IndexBase):\n    @property\n    def calendar_list(self) -> List[pd.Timestamp]:\n        \"\"\"get history trading date\n\n        Returns\n        -------\n            calendar list\n        \"\"\"\n        _calendar = getattr(self, \"_calendar_list\", None)\n        if not _calendar:\n            _calendar = get_calendar_list(bench_code=self.index_name.upper())\n            setattr(self, \"_calendar_list\", _calendar)\n        return _calendar\n","sourceCodeStart":27,"sourceCodeEnd":63,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/scripts/data_collector/cn_index/collector.py#L27-L63","documentation":"Raised by retry_request in the cn_index (CSI index) collector when the HTTP response status is not 200 and not in exclude_status. It is wrapped in @deco_retry, so the request is retried several times before the ValueError propagates, and the message includes the status code and URL for diagnosis.","triggerScenarios":"Fetching CSI 300 / index constituent pages (csindex.com.cn endpoints) where the server returns 403/418 (bot blocking), 5xx downtime, or a redirect; the URL is built from the instrument's index code query.","commonSituations":"Running the cn_index collector from an IP/datacenter the site rate-limits or blocks; transient server outage; missing/wrong index code producing 404; too-frequent runs without backoff.","solutions":["Retry later / with delays — deco_retry already retries; if persistently 403, run from a residential network or reduce frequency, and set a realistic User-Agent (the code sends a Chrome UA).","Verify the index code/url manually in a browser or with curl to see the actual status code.","If the status is a known-acceptable condition for your run, pass it via exclude_status when calling retry_request in custom code.","Upgrade qlib — collector scripts have been patched over time for upstream site changes."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"resp = requests.get(url, headers=REQ_HEADERS, timeout=30)\nif resp.status_code == 200:\n    data = resp.json()\nelse:\n    raise RuntimeError(f\"cn_index endpoint returned {resp.status_code}\")","typeGuard":null,"tryCatchPattern":"from qlib.utils import deco_retry  # or your own retry wrapper\n@deco_retry\ndef fetch(url):\n    resp = requests.get(url, headers=REQ_HEADERS, timeout=30)\n    if resp.status_code != 200:\n        raise ValueError(f\"response status: {resp.status_code}, url={url}\")\n    return resp","preventionTips":["Run the cn_index collector at low frequency and from non-datacenter IPs.","Check the index URL manually when collection fails before re-running.","Wrap calls in retry with exponential backoff for transient 5xx."],"tags":["qlib","data-collector","http","cn-index","network"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}