microsoft/qlib · warning · ValueError

request error

Error message

request error

What it means

Raised by FundCollector.get_data_from_remote when the eastmoney fund API responds with a non-200 status. The request uses INDEX_BENCH_URL with a referer header; a non-200 is considered a request failure. Note the outer try logs a warning with '{symbol}-{interval}-{start}-{end}' and returns None rather than propagating.

Source

Thrown at scripts/data_collector/fund/collector.py:113

    @property
    @abc.abstractmethod
    def _timezone(self):
        raise NotImplementedError("rewrite get_timezone")

    @staticmethod
    def get_data_from_remote(symbol, interval, start, end):
        error_msg = f"{symbol}-{interval}-{start}-{end}"

        try:
            # TODO: numberOfHistoricalDaysToCrawl should be bigger enough
            url = INDEX_BENCH_URL.format(
                index_code=symbol, numberOfHistoricalDaysToCrawl=10000, startDate=start, endDate=end
            )
            resp = requests.get(url, headers={"referer": "http://fund.eastmoney.com/110022.html"}, timeout=None)

            if resp.status_code != 200:
                raise ValueError("request error")

            data = json.loads(resp.text.split("(")[-1].split(")")[0])

            # Some funds don't show the net value, example: http://fundf10.eastmoney.com/jjjz_010288.html
            SYType = data["Data"]["SYType"]
            if SYType in {"每万份收益", "每百份收益", "每百万份收益"}:
                raise ValueError("The fund contains 每*份收益")

            # TODO: should we sort the value by datetime?
            _resp = pd.DataFrame(data["Data"]["LSJZList"])

            if isinstance(_resp, pd.DataFrame):
                return _resp.reset_index()
        except Exception as e:
            logger.warning(f"{error_msg}:{e}")

    def get_data(
        self, symbol: str, interval: str, start_datetime: pd.Timestamp, end_datetime: pd.Timestamp

View on GitHub (pinned to 79633dd950)

Solutions

1. Check the logged warning for the failing symbol and test the URL in a browser/curl with the referer header to inspect the real status.
2. Slow down / add sleeps between requests or reduce max_workers to avoid throttling.
3. Confirm fund codes are valid eastmoney codes (6 digits typically); drop retired funds from the instrument list.
4. Retry later — eastmoney intermittently blocks datacenter IPs.

resp = requests.get(url, headers={"referer": "http://fund.eastmoney.com/110022.html"}, timeout=30)
if resp.status_code != 200:
    raise RuntimeError(f"eastmoney returned {resp.status_code} for {symbol}; retry later or reduce concurrency")

try:
    df = FundCollector.get_data_from_remote(symbol, interval, start, end)
except Exception:
    df = None  # collector already logs the warning; skip symbol and continue
if df is None:
    logger.warning(f"skipping {symbol}: no data returned")

When it happens

Trigger: Requesting historical NAV for a fund code while fund.eastmoney.com returns 403/302/5xx (anti-bot, invalid referer, rate limiting) or the fund code is malformed yielding an error page.

Common situations: Running the fund collector at high concurrency triggering eastmoney throttling; network/proxy issues; fund codes scraped from a stale instrument list that no longer resolve.

Related errors

• The fund contains 每*份收益
• response status: {_status}, url={url}
• request error
• interval error: {self.interval}
• source_dir and target_dir cannot be None

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/2c190770350a3c71. Report an issue: GitHub.