DIYgod/RSSHub · error · TypeError

TechFlow API still returned an anti-crawler challenge after

Error message

TechFlow API still returned an anti-crawler challenge after retry.

What it means

Thrown by requestApi after it solved the TechFlow 'acw_sc__v2' anti-crawler JS challenge, set the derived cookie, and re-issued the request, but the second response body is still a string (i.e. the challenge HTML page, not JSON). It means the cookie-based bypass failed to convince the upstream WAF that the client is a real browser.

Source

Thrown at lib/routes/techflowpost/utils.ts:97

        got.get(`${apiRootUrl}${endpoint}`, {
            searchParams,
            headers: getHeaders(referer),
        });

    const { data } = await request();
    if (typeof data !== 'string') {
        return data as T;
    }

    const cookie = getAcwScV2Cookie(data);
    if (!cookie) {
        throw new Error('TechFlow API returned an unexpected non-JSON response.');
    }

    acwScV2Cookie = cookie;
    const retryResponse = await request();
    if (typeof retryResponse.data === 'string') {
        throw new TypeError('TechFlow API still returned an anti-crawler challenge after retry.');
    }

    return retryResponse.data as T;
}

function getPictureUrl(picture?: string) {
    if (!picture) {
        return;
    }
    return new URL(picture, uploadRootUrl).href;
}

function getCategories(article: Article) {
    return [...new Set([article.category?.name, ...(article.labels?.map((label) => label.label) ?? [])].filter(Boolean))] as string[];
}

function getArticleItem(article: Article, content?: string): DataItem {
    const link = `${rootUrl}/${locale}/article/${article.id}`;

View on GitHub (pinned to bed535e087)

Solutions

1. Retry the feed request after a longer delay (minutes) from a residential/clean IP, since the failure is usually IP-reputation driven.
2. Verify getHeaders() still sends a realistic browser User-Agent and the correct Referer (the per-endpoint link) that the WAF expects.
3. Check whether getAcwScV2ByArg1 still matches the current obfuscation: open the challenge HTML in a real browser, extract arg1, and compare the computed acw_sc__v2 against what the browser sets.
4. If persistent, route RSSHub egress through a proxy the WAF tolerates, or cache a working acw_sc__v2 cookie from a browser session and inject it into the initial request headers.

Example fix

// before
acwScV2Cookie = cookie;
const retryResponse = await request();
if (typeof retryResponse.data === 'string') {
    throw new TypeError('TechFlow API still returned an anti-crawler challenge after retry.');
}

// after: one bounded retry with a delay, then a clearer error
acwScV2Cookie = cookie;
let retryResponse;
for (let attempt = 0; attempt < 2; attempt++) {
    await new Promise((r) => setTimeout(r, 1000 * (attempt + 1)));
    retryResponse = await request();
    if (typeof retryResponse.data !== 'string') {
        return retryResponse.data as T;
    }
}
throw new TypeError('TechFlow anti-crawler challenge persists; IP likely flagged or acw_sc__v2 logic outdated.');

// No caller-side data check prevents a WAF challenge, but you can pre-flight reachability
async function looksLikeJsonApi(url: string): Promise<boolean> {
    try {
        const r = await fetch(url, { headers: { Accept: 'application/json' } });
        const ct = r.headers.get('content-type') ?? '';
        return ct.includes('application/json');
    } catch {
        return false;
    }
}
// if false, expect a challenge; back off before hitting requestApi

// detect an anti-crawler challenge body so callers can retry instead of throwing
function isAntiCrawlerChallenge(data: unknown): data is string {
    return typeof data === 'string' && /var arg1=/.test(data);
}

try {
    return await requestApi<Article>(endpoint, referer, params);
} catch (e) {
    if (e instanceof TypeError && /anti-crawler challenge after retry/.test(e.message)) {
        await sleep(60_000); // WAF reputation usually recovers within minutes
        return await requestApi<Article>(endpoint, referer, params);
    }
    throw e;
}

When it happens

Trigger: The upstream site (TechFlow) returns an HTML page embedding 'var arg1=...' the first time; getAcwScV2Cookie derives acw_sc__v2 and the route retries. If the server still serves the obfuscated challenge HTML on retry (IP flagged, challenge rotated, UA rejected, or arg1 logic outdated), retryResponse.data stays a string and this TypeError fires.

Common situations: Running RSSHub from a datacenter/cloud IP that the WAF has greylisted; the acw_sc__v2 derivation algorithm being patched server-side so the generated cookie is rejected; missing/changed browser-like headers (Referer/User-Agent) that the WAF cross-checks; rate-limiting after many requests.

Related errors

• TechFlow API returned an unexpected non-JSON response.
• 访问量过大，触发验证码
• 文章列表获取失败，可能是被临时限制了访问，请稍后重试 ${JSON.stringify(resp.data)}
• Cannot get the redirect link
• Failed to fetch thread data

AI-assisted analysis of DIYgod/RSSHub@bed535e087 (2026-08-12). Data as JSON: /api/errors/b875ccac95868b4c. Report an issue: GitHub.