DIYgod/RSSHub · error · Error

Failed to retrieve JSON beatmap info from osu! website

Error message

Failed to retrieve JSON beatmap info from osu! website

What it means

Thrown when the osu! beatmapsets page is fetched but the embedded #json-beatmaps script element is absent or empty, causing the fallback JSON '{"beatmapsets": undefined}' to parse and leave beatmapsets undefined. The route depends on osu! injecting this server-rendered JSON blob; losing it means the page can't be scraped.

Source

Thrown at lib/routes/osu/beatmaps/latest-ranked.tsx:206

    const difficultyLimits = searchParams.getAll('difficultyLimit');
    const modeInTitle = searchParams.get('modeInTitle') ?? 'true'; // show mode name in title, default to true.

    // fetch beatmap JSON info from website within cache
    let beatmapsetList = (await cache.tryGet(
        'https://osu.ppy.sh/beatmapsets:JSON',
        async () => {
            const link = 'https://osu.ppy.sh/beatmapsets';

            const response = await got.get(link);
            const $ = load(response.data);

            const beatmapInfo = JSON.parse($('#json-beatmaps').text() ?? '{"beatmapsets": undefined}');

            const beatmapList: BeatmapsetInfo[] = beatmapInfo.beatmapsets;

            // Failed to fetch, raise error
            if (beatmapList === undefined) {
                throw new Error('Failed to retrieve JSON beatmap info from osu! website');
            }

            return beatmapList;
        },
        config.cache.routeExpire,
        false
    )) as BeatmapsetInfo[];

    // Sort beatmap by difficultyRate.desc
    // This step is necessary even if difficultyLimit not enabled, since we want the beatmap
    // in RSS description sorted when displayed
    for (const item of beatmapsetList) {
        item.beatmaps.sort((a, b) => a.difficulty_rating - b.difficulty_rating);
    }

    // filter beatmapset types
    // Note:
    // One Osu beatmapset could actually contains several beatmaps with different game mode.

View on GitHub (pinned to bed535e087)

Solutions

1. Open https://osu.ppy.sh/beatmapsets and confirm a <script id="json-beatmaps"> element still exists with JSON content.
2. Flush the cache key 'https://osu.ppy.sh/beatmapsets:JSON' so the next request re-fetches fresh HTML.
3. If osu! removed the element, migrate the route to the official osu! API (requires an API key) or to the new DOM structure.
4. Use config.trueUA / a browser-like header to avoid receiving a stripped or challenge page.

Example fix

// before
const beatmapInfo = JSON.parse($('#json-beatmaps').text() ?? '{"beatmapsets": undefined}');
const beatmapList: BeatmapsetInfo[] = beatmapInfo.beatmapsets;
if (beatmapList === undefined) {
    throw new Error('Failed to retrieve JSON beatmap info from osu! website');
}

// after — fail with the actual cause (missing/empty element) and don't swallow it into undefined
const raw = $('#json-beatmaps').text();
if (!raw) {
    throw new Error('Failed to retrieve JSON beatmap info from osu! website: #json-beatmaps element is empty or absent');
}

// Probe the beatmapsets page for the embedded JSON element before scraping.
async function beatmapsJsonAvailable(): Promise<boolean> {
    const html = await got.get('https://osu.ppy.sh/beatmapsets').then((r) => r.data);
    const $ = load(html);
    return $('#json-beatmaps').text().length > 0;
}

const hasBeatmapsets = (o: any): o is { beatmapsets: unknown[] } =>
    o !== null && typeof o === 'object' && Array.isArray(o.beatmapsets);

try {
    beatmapsetList = await cache.tryGet('https://osu.ppy.sh/beatmapsets:JSON', fetcher, config.cache.routeExpire, false);
} catch (e) {
    if (e instanceof Error && /Failed to retrieve JSON beatmap info/.test(e.message)) {
        await cache.delete?.('https://osu.ppy.sh/beatmapsets:JSON'); // not all caches expose delete
        // one retry with a browser UA
        beatmapsetList = await cache.tryGet('https://osu.ppy.sh/beatmapsets:JSON', fetcher, config.cache.routeExpire, true);
    } else throw e;
}

When it happens

Trigger: GET https://osu.ppy.sh/beatmapsets returns HTML whose #json-beatmaps element is missing/empty (or got blocked and returned a Cloudflare challenge page). JSON.parse of undefined-text falls back to the literal '{"beatmapsets": undefined}', so beatmapInfo.beatmapsets is undefined and the guard fires.

Common situations: osu! redesigned the beatmapsets page and removed/renamed the #json-beatmaps element; the instance IP got a Cloudflare interstitial or a localized page without the JSON; the cached entry under 'https://osu.ppy.sh/beatmapsets:JSON' was populated during an outage and is served stale until routeExpire elapses.

Related errors

• Cannot find the script with data-iso-key="_0"
• Unable to extract creator ID
• JavaScript file not found.
• 无法解析页面数据，请检查漫画 ID 是否正确或页面结构是否变动
• 无法解析页面 HTML 数据，可能触发了反爬策略或页面结构巨变

AI-assisted analysis of DIYgod/RSSHub@bed535e087 (2026-08-12). Data as JSON: /api/errors/cfe320734a9fce0f. Report an issue: GitHub.