DIYgod/RSSHub · error · Error

Failed to extract Algolia credentials from iapp.org

Error message

Failed to extract Algolia credentials from iapp.org

What it means

Thrown by the IAPP news route when the handler cannot extract Algolia `appID` and `apiKey` from the iapp.org homepage HTML. The route scrapes the page's `<script>` tags looking for a specific escaped JSON pattern (`\"appID\":\"...\",\"apiKey\":\"...\"`). If iapp.org changes its JavaScript bundling, renames the keys, changes the escaping, or stops embedding credentials client-side, the regex will not match.

Source

Thrown at lib/routes/iapp/news.ts:46

    const baseUrl = 'https://iapp.org';
    const link = `${baseUrl}/news`;

    const { appId, apiKey, description } = await cache.tryGet('iapp:algolia-credentials', async () => {
        const html = await ofetch(link);
        const $ = load(html);
        let appId: string | undefined;
        let apiKey: string | undefined;
        $('script').each((_, el) => {
            const text = $(el).text();
            const match = text.match(/\\"appID\\":\\"(\w+)\\",\\"apiKey\\":\\"(\w+)\\"/);
            if (match) {
                appId = match[1];
                apiKey = match[2];
                return false;
            }
        });
        if (!appId || !apiKey) {
            throw new Error('Failed to extract Algolia credentials from iapp.org');
        }
        return {
            appId,
            apiKey,
            description: $('meta[name="description"]').attr('content'),
        };
    });

    const response = await ofetch(`https://${appId.toLowerCase()}-dsn.algolia.net/1/indexes/*/queries`, {
        method: 'POST',
        query: {
            'x-algolia-api-key': apiKey,
            'x-algolia-application-id': appId,
        },
        headers: {
            Referer: `${baseUrl}/`,
        },
        body: {

View on GitHub (pinned to bed535e087)

Solutions

Open `https://iapp.org/news` in a browser, view page source, and search for `appID` and `apiKey` to see the current format.
Update the regex at line 38 to match the new format (e.g. adjust escape patterns, handle different quote styles, or switch to a JSON parsing approach).
Clear the `iapp:algolia-credentials` cache key after deploying a fix.
If credentials are no longer client-side, switch to a server-side API endpoint or a different data source.

Example fix

// before (brittle regex tied to specific escaping)
const match = text.match(/\\"appID\\":\\"(\\w+)\\",\\"apiKey\\":\\"(\\w+)\\"/);

// after (more robust: handle multiple escaping styles)
const match = text.match(/appID[":\\\s]+([A-Z0-9]+)["\\\s,]+apiKey[":\\\s]+([a-f0-9]+)/i);

Defensive patterns

Strategy: try-catch

Type guard

function hasAlgoliaCredentials(html: string): boolean {
    return /appID/.test(html) && /apiKey/.test(html);
}

Try / catch

try {
    const $ = load(html);
    // ... regex extraction
    if (!appId || !apiKey) {
        throw new Error('Failed to extract Algolia credentials from iapp.org');
    }
} catch (err) {
    throw new Error(`iapp.org credential extraction failed: ${err instanceof Error ? err.message : String(err)}`);
}

Prevention

Cache extracted credentials with a TTL to reduce scraping frequency.
Make the extraction regex robust against different escaping styles.
Periodically verify the iapp.org homepage structure hasn't changed.
Fall back to an alternative data source if client-side credentials are removed.

When it happens

Trigger: The iapp.org homepage no longer embeds Algolia credentials in the expected format. The credentials are cached under `iapp:algolia-credentials`, so the error first appears after the cache expires. The regex at line 38 is brittle and depends on the exact escaping of the JSON within the script tag.

Common situations: IAPP updates their Next.js/Astro build pipeline changing how the Algolia config is serialized, they move to server-side search proxying (removing client-side credentials), they rename the config keys, or the escaping changes from `\"` to `"` or template literals.

Related errors

AI-assisted analysis of DIYgod/RSSHub@bed535e087 (2026-08-12). Data as JSON: /api/errors/d74b3afb4ad05dba. Report an issue: GitHub.