{"record":{"id":"cfe320734a9fce0f","repo":"DIYgod/RSSHub","slug":"failed-to-retrieve-json-beatmap-info-from-osu-web","errorCode":null,"errorMessage":"Failed to retrieve JSON beatmap info from osu! website","messagePattern":"Failed to retrieve JSON beatmap info from osu! website","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"lib/routes/osu/beatmaps/latest-ranked.tsx","lineNumber":206,"sourceCode":"    const difficultyLimits = searchParams.getAll('difficultyLimit');\n    const modeInTitle = searchParams.get('modeInTitle') ?? 'true'; // show mode name in title, default to true.\n\n    // fetch beatmap JSON info from website within cache\n    let beatmapsetList = (await cache.tryGet(\n        'https://osu.ppy.sh/beatmapsets:JSON',\n        async () => {\n            const link = 'https://osu.ppy.sh/beatmapsets';\n\n            const response = await got.get(link);\n            const $ = load(response.data);\n\n            const beatmapInfo = JSON.parse($('#json-beatmaps').text() ?? '{\"beatmapsets\": undefined}');\n\n            const beatmapList: BeatmapsetInfo[] = beatmapInfo.beatmapsets;\n\n            // Failed to fetch, raise error\n            if (beatmapList === undefined) {\n                throw new Error('Failed to retrieve JSON beatmap info from osu! website');\n            }\n\n            return beatmapList;\n        },\n        config.cache.routeExpire,\n        false\n    )) as BeatmapsetInfo[];\n\n    // Sort beatmap by difficultyRate.desc\n    // This step is necessary even if difficultyLimit not enabled, since we want the beatmap\n    // in RSS description sorted when displayed\n    for (const item of beatmapsetList) {\n        item.beatmaps.sort((a, b) => a.difficulty_rating - b.difficulty_rating);\n    }\n\n    // filter beatmapset types\n    // Note:\n    // One Osu beatmapset could actually contains several beatmaps with different game mode.","sourceCodeStart":188,"sourceCodeEnd":224,"githubUrl":"https://github.com/DIYgod/RSSHub/blob/bed535e0879dc71c5aff6f1e7bd1ac21ede40115/lib/routes/osu/beatmaps/latest-ranked.tsx#L188-L224","documentation":"Thrown when the osu! beatmapsets page is fetched but the embedded #json-beatmaps script element is absent or empty, causing the fallback JSON '{\"beatmapsets\": undefined}' to parse and leave beatmapsets undefined. The route depends on osu! injecting this server-rendered JSON blob; losing it means the page can't be scraped.","triggerScenarios":"GET https://osu.ppy.sh/beatmapsets returns HTML whose #json-beatmaps element is missing/empty (or got blocked and returned a Cloudflare challenge page). JSON.parse of undefined-text falls back to the literal '{\"beatmapsets\": undefined}', so beatmapInfo.beatmapsets is undefined and the guard fires.","commonSituations":"osu! redesigned the beatmapsets page and removed/renamed the #json-beatmaps element; the instance IP got a Cloudflare interstitial or a localized page without the JSON; the cached entry under 'https://osu.ppy.sh/beatmapsets:JSON' was populated during an outage and is served stale until routeExpire elapses.","solutions":["Open https://osu.ppy.sh/beatmapsets and confirm a <script id=\"json-beatmaps\"> element still exists with JSON content.","Flush the cache key 'https://osu.ppy.sh/beatmapsets:JSON' so the next request re-fetches fresh HTML.","If osu! removed the element, migrate the route to the official osu! API (requires an API key) or to the new DOM structure.","Use config.trueUA / a browser-like header to avoid receiving a stripped or challenge page."],"exampleFix":"// before\nconst beatmapInfo = JSON.parse($('#json-beatmaps').text() ?? '{\"beatmapsets\": undefined}');\nconst beatmapList: BeatmapsetInfo[] = beatmapInfo.beatmapsets;\nif (beatmapList === undefined) {\n    throw new Error('Failed to retrieve JSON beatmap info from osu! website');\n}\n\n// after — fail with the actual cause (missing/empty element) and don't swallow it into undefined\nconst raw = $('#json-beatmaps').text();\nif (!raw) {\n    throw new Error('Failed to retrieve JSON beatmap info from osu! website: #json-beatmaps element is empty or absent');\n}","handlingStrategy":"retry","validationCode":"// Probe the beatmapsets page for the embedded JSON element before scraping.\nasync function beatmapsJsonAvailable(): Promise<boolean> {\n    const html = await got.get('https://osu.ppy.sh/beatmapsets').then((r) => r.data);\n    const $ = load(html);\n    return $('#json-beatmaps').text().length > 0;\n}","typeGuard":"const hasBeatmapsets = (o: any): o is { beatmapsets: unknown[] } =>\n    o !== null && typeof o === 'object' && Array.isArray(o.beatmapsets);","tryCatchPattern":"try {\n    beatmapsetList = await cache.tryGet('https://osu.ppy.sh/beatmapsets:JSON', fetcher, config.cache.routeExpire, false);\n} catch (e) {\n    if (e instanceof Error && /Failed to retrieve JSON beatmap info/.test(e.message)) {\n        await cache.delete?.('https://osu.ppy.sh/beatmapsets:JSON'); // not all caches expose delete\n        // one retry with a browser UA\n        beatmapsetList = await cache.tryGet('https://osu.ppy.sh/beatmapsets:JSON', fetcher, config.cache.routeExpire, true);\n    } else throw e;\n}","preventionTips":["Use config.trueUA to avoid Cloudflare challenge pages.","Prefer the official osu! API (with API key) over scraping where possible.","Don't cache a failure result — only cache when #json-beatmaps is non-empty."],"tags":["scraping","upstream-change","anti-bot","dom-parsing","osu"],"backgroundTag":null,"analyzedSha":"bed535e0879dc71c5aff6f1e7bd1ac21ede40115","analyzedAt":"2026-08-12T19:29:35.364Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}