binary-husky/gpt_academic · error · ValueError

获取RSS feed失败，状态码: {feed.status}

Error message

获取RSS feed失败，状态码: {feed.status}

What it means

In the RSS-based category fetch, feedparser.parse(feed_url) is checked for an HTTP status attribute; any status other than 200 raises ValueError('获取RSS feed失败，状态码: ...'). This happens when rss.arxiv.org answers with an HTTP error (403 rate-limit, 5xx outage, redirect to a block page) rather than the feed XML.

Source

Thrown at crazy_functions/review_fns/data_sources/arxiv_source.py:445

        Raises:
            ValueError: 如果类别无效
        """
        try:
            # 处理类别格式
            # 1. 转换为小写
            # 2. 确保多个类别之间使用+连接
            category = category.lower().replace(' ', '+')

            # 构建RSS feed URL
            feed_url = f"https://rss.arxiv.org/rss/{category}"
            print(f"正在获取RSS feed: {feed_url}")  # 添加调试信息

            feed = feedparser.parse(feed_url)

            # 检查feed是否有效
            if hasattr(feed, 'status') and feed.status != 200:
                raise ValueError(f"获取RSS feed失败，状态码: {feed.status}")

            if not feed.entries:
                print(f"警告：未在feed中找到任何条目")  # 添加调试信息
                print(f"Feed标题: {feed.feed.title if hasattr(feed, 'feed') else '无标题'}")
                raise ValueError(f"无效的arXiv类别或未找到论文: {category}")

            if debug:
                # 调试模式：只获取5篇最新论文
                search = arxiv.Search(
                    query=f'cat:{category}',
                    sort_by=arxiv.SortCriterion.SubmittedDate,
                    sort_order=arxiv.SortOrder.Descending,
                    max_results=5
                )
                results = list(self.client.results(search))
                return [self._parse_paper_data(result) for result in results]

            # 正常模式：获取所有新论文

View on GitHub (pinned to d6bde0fa54)

Solutions

1. Wait and retry with backoff (minutes, not seconds) — 403/429 from rss.arxiv.org are almost always rate limiting.
2. Reduce polling frequency and cache results; arXiv RSS updates roughly daily.
3. Run from a residential/different IP or configure the proxies setting if the environment blocks rss.arxiv.org.
4. Check https://status.arxiv.org for feed outages when the status is 5xx.

Example fix

# before
feed = feedparser.parse(feed_url)
if hasattr(feed, 'status') and feed.status != 200:
    raise ValueError(f'获取RSS feed失败，状态码: {feed.status}')

# after
for attempt in range(3):
    feed = feedparser.parse(feed_url)
    status = getattr(feed, 'status', None)
    if status == 200 and feed.entries:
        break
    time.sleep(60 * (attempt + 1))
else:
    raise ValueError(f'获取RSS feed失败，状态码: {status}')

for attempt in range(3):
    try:
        papers = await source.fetch_by_category(cat)
        break
    except ValueError as e:
        if '状态码' in str(e) and attempt < 2:
            await asyncio.sleep(60 * (attempt + 1))  # arXiv rate-limits: back off
            continue
        raise

When it happens

Trigger: Calling the category/new-paper listing method (e.g. fetch_latest by category like 'cs.AI') while rss.arxiv.org returns 403/429 (aggressive polling or shared cloud IP), 5xx during arXiv maintenance, or when a proxy/VPN intercepts the request and answers with an error page status.

Common situations: Scripts polling the RSS feed in a tight loop and hitting arXiv rate limits; running from datacenter IPs (AWS/GCP) that arXiv throttles; corporate proxies that rewrite responses; temporary rss.arxiv.org outages.

Related errors

• 无效的arXiv类别或未找到论文: {category}
• 所有可用镜像站点均无法完成下载
• 在线搜索失败！\n{Exceptions}
• Searxng（在线搜索服务）当前使用人数太多，请稍后。
• 无法下载资源{txt}，请检查。

AI-assisted analysis of binary-husky/gpt_academic@d6bde0fa54 (2026-08-14). Data as JSON: /api/errors/e80273195d86f24b. Report an issue: GitHub.