microsoft/qlib · error · ValueError

response status: {_status}, url={url}

Error message

response status: {_status}, url={url}

What it means

Raised by retry_request in the cn_index (CSI index) collector when the HTTP response status is not 200 and not in exclude_status. It is wrapped in @deco_retry, so the request is retried several times before the ValueError propagates, and the message includes the status code and URL for diagnosis.

Source

Thrown at scripts/data_collector/cn_index/collector.py:45

)


INDEX_CHANGES_URL = "https://www.csindex.com.cn/csindex-home/search/search-content?lang=cn&searchInput=%E5%85%B3%E4%BA%8E%E8%B0%83%E6%95%B4%E6%B2%AA%E6%B7%B1300%E5%92%8C%E4%B8%AD%E8%AF%81%E9%A6%99%E6%B8%AF100%E7%AD%89%E6%8C%87%E6%95%B0%E6%A0%B7%E6%9C%AC&pageNum={page_num}&pageSize={page_size}&sortField=date&dateRange=all&contentType=announcement"

REQ_HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36 Edg/91.0.864.48"
}


@deco_retry
def retry_request(url: str, method: str = "get", exclude_status: List = None):
    if exclude_status is None:
        exclude_status = []
    method_func = getattr(requests, method)
    _resp = method_func(url, headers=REQ_HEADERS, timeout=None)
    _status = _resp.status_code
    if _status not in exclude_status and _status != 200:
        raise ValueError(f"response status: {_status}, url={url}")
    return _resp


class CSIIndex(IndexBase):
    @property
    def calendar_list(self) -> List[pd.Timestamp]:
        """get history trading date

        Returns
        -------
            calendar list
        """
        _calendar = getattr(self, "_calendar_list", None)
        if not _calendar:
            _calendar = get_calendar_list(bench_code=self.index_name.upper())
            setattr(self, "_calendar_list", _calendar)
        return _calendar

View on GitHub (pinned to 79633dd950)

Solutions

1. Retry later / with delays — deco_retry already retries; if persistently 403, run from a residential network or reduce frequency, and set a realistic User-Agent (the code sends a Chrome UA).
2. Verify the index code/url manually in a browser or with curl to see the actual status code.
3. If the status is a known-acceptable condition for your run, pass it via exclude_status when calling retry_request in custom code.
4. Upgrade qlib — collector scripts have been patched over time for upstream site changes.

resp = requests.get(url, headers=REQ_HEADERS, timeout=30)
if resp.status_code == 200:
    data = resp.json()
else:
    raise RuntimeError(f"cn_index endpoint returned {resp.status_code}")

from qlib.utils import deco_retry  # or your own retry wrapper
@deco_retry
def fetch(url):
    resp = requests.get(url, headers=REQ_HEADERS, timeout=30)
    if resp.status_code != 200:
        raise ValueError(f"response status: {resp.status_code}, url={url}")
    return resp

When it happens

Trigger: Fetching CSI 300 / index constituent pages (csindex.com.cn endpoints) where the server returns 403/418 (bot blocking), 5xx downtime, or a redirect; the URL is built from the instrument's index code query.

Common situations: Running the cn_index collector from an IP/datacenter the site rate-limits or blocks; transient server outage; missing/wrong index code producing 404; too-frequent runs without backoff.

Related errors

• request error
• request error
• source_dir and target_dir cannot be None
• Unsupported freq: {freq}
• interval error: {self.interval}

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/063b32cdaac6d40c. Report an issue: GitHub.