{"record":{"id":"f281337d59786736","repo":"microsoft/qlib","slug":"request-error-f28133","errorCode":null,"errorMessage":"request error","messagePattern":"request error","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/data_collector/utils.py","lineNumber":306,"sourceCode":"\ndef get_us_stock_symbols(qlib_data_path: [str, Path] = None) -> list:\n    \"\"\"get US stock symbols\n\n    Returns\n    -------\n        stock symbols\n    \"\"\"\n    import akshare as ak  # pylint: disable=C0415\n\n    global _US_SYMBOLS  # pylint: disable=W0603\n\n    @deco_retry\n    def _get_eastmoney():\n        df = ak.get_us_stock_name()\n        _symbols = df[\"symbol\"].to_list()\n\n        if len(_symbols) < 8000:\n            raise ValueError(\"request error\")\n\n        return _symbols\n\n    @deco_retry\n    def _get_nasdaq():\n        _res_symbols = []\n        for _name in [\"otherlisted\", \"nasdaqtraded\"]:\n            url = f\"ftp://ftp.nasdaqtrader.com/SymbolDirectory/{_name}.txt\"\n            df = pd.read_csv(url, sep=\"|\")\n            df = df.rename(columns={\"ACT Symbol\": \"Symbol\"})\n            _symbols = df[\"Symbol\"].dropna()\n            _symbols = _symbols.str.replace(\"$\", \"-P\", regex=False)\n            _symbols = _symbols.str.replace(\".W\", \"-WT\", regex=False)\n            _symbols = _symbols.str.replace(\".U\", \"-UN\", regex=False)\n            _symbols = _symbols.str.replace(\".R\", \"-RI\", regex=False)\n            _symbols = _symbols.str.replace(\".\", \"-\", regex=False)\n            _res_symbols += _symbols.unique().tolist()\n        return _res_symbols","sourceCodeStart":288,"sourceCodeEnd":324,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/scripts/data_collector/utils.py#L288-L324","documentation":"Raised inside _get_eastmoney of get_us_symbols (scripts/data_collector/utils.py:306) when ak.get_us_stock_name() returns fewer than 8000 symbols. Like 590, it is a completeness sentinel: the akshare call 'succeeded' but the list is implausibly small for the US market, so the scraper treats it as a failed/incomplete fetch. The @deco_retry wrapper retries it before the error propagates.","triggerScenarios":"Calling get_us_symbols() when akshare's US stock-name endpoint returns a truncated table (upstream throttling, layout change breaking akshare's parser, or an old akshare version returning partial data), leaving len < 8000.","commonSituations":"Outdated akshare package whose scraping selectors no longer match the eastmoney page; rate limiting after repeated runs in CI; akshare API renames (ak.get_us_stock_name may itself fail on some versions) surfacing as short lists.","solutions":["Upgrade akshare (pip install -U akshare) — scraper drift in old versions is the usual cause of truncated tables.","Retry after a delay; @deco_retry already retries, so add a coarser outer retry/backoff.","Verify manually: python -c \"import akshare as ak; print(len(ak.get_us_stock_name()))\" — if persistently < 8000, the akshare source needs fixing.","get_us_symbols falls back to _get_nasdaq()/_get_nyse() sources in the same function; ensure those network paths also work so the combined list is healthy."],"exampleFix":"# before\nsymbols = get_us_symbols()  # eastmoney path keeps returning short list\n\n# after\npip install -U akshare  # then re-run\nsymbols = get_us_symbols()","handlingStrategy":"retry","validationCode":"import akshare as ak\ndf = ak.get_us_stock_name()\nif len(df) < 8000:\n    raise SystemExit('akshare returned a truncated US list; upgrade akshare or retry')","typeGuard":null,"tryCatchPattern":"try:\n    symbols = get_us_symbols()\nexcept ValueError as e:\n    if 'request error' in str(e):\n        time.sleep(60)\n        symbols = get_us_symbols()\n    else:\n        raise","preventionTips":["Pin/upgrade akshare deliberately — its scrapers drift with the upstream site.","Cache the resulting symbol set to disk so downstream steps do not re-hit the source."],"tags":["us-symbols","akshare","eastmoney","completeness-check"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}