jackwener/OpenCLI · error · CommandExecutionError

WeRead search page request failed: HTTP ${resp.status}

Error message

WeRead search page request failed: HTTP ${resp.status}

What it means

Thrown by loadSearchHtmlEntries when the WeRead search page request completes but responds with a non-2xx status (resp.ok is false). The library surfaces the HTTP status code in a CommandExecutionError because the HTML result list cannot be scraped from an error page (403 anti-bot, 404 route change, 5xx outage).

Source

Thrown at clis/weread/search.js:113

/**
 * Extract rendered search result reader URLs from the server-rendered search page.
 * The public JSON API still returns bookId, but the current web app links results
 * through /web/reader/<opaque-id> rather than /web/bookDetail/<bookId>.
 */
async function loadSearchHtmlEntries(query) {
    const url = new URL('/web/search/books', WEREAD_WEB_ORIGIN);
    url.searchParams.set('keyword', query);
    let resp;
    try {
        resp = await fetch(url.toString(), {
            headers: { 'User-Agent': WEREAD_UA },
        });
    }
    catch (error) {
        throw new CommandExecutionError(`Failed to fetch WeRead search page: ${error instanceof Error ? error.message : String(error)}`);
    }
    if (!resp.ok) {
        throw new CommandExecutionError(`WeRead search page request failed: HTTP ${resp.status}`);
    }
    const html = await resp.text();
    const items = Array.from(html.matchAll(/<li[^>]*class="wr_bookList_item"[^>]*>([\s\S]*?)<\/li>/g));
    return items.map((match) => {
        const chunk = match[1];
        const hrefMatch = chunk.match(/<a[^>]*href="([^"]+)"[^>]*class="wr_bookList_item_link"[^>]*>|<a[^>]*class="wr_bookList_item_link"[^>]*href="([^"]+)"[^>]*>/);
        const titleMatch = chunk.match(/<p[^>]*class="wr_bookList_item_title"[^>]*>([\s\S]*?)<\/p>/);
        const authorMatch = chunk.match(/<p[^>]*class="wr_bookList_item_author"[^>]*>([\s\S]*?)<\/p>/);
        const href = hrefMatch?.[1] || hrefMatch?.[2] || '';
        const title = decodeHtmlText(titleMatch?.[1] || '');
        const author = decodeHtmlText(authorMatch?.[1] || '');
        return {
            author,
            url: href ? new URL(href, WEREAD_WEB_ORIGIN).toString() : '',
            title,
        };
    }).filter((item) => item.url && item.title);
}

View on GitHub (pinned to 49907e53dc)

Solutions

1. Retry after a delay with fewer requests — 403/429 usually indicate rate limiting or anti-bot throttling.
2. Verify the route still exists: open https://weread.qq.com/web/search/books?keyword=test in a browser; a 404 after a redesign means the scraper regex/URL must be updated.
3. Capture the response body on failure (resp.text() before throwing) to see whether WeRead returns a captcha/challenge page.
4. Add realistic headers (Accept, Accept-Language) alongside WEREAD_UA to reduce WAF rejections.
5. Check status specifics: 5xx → wait for WeRead to recover; 403/429 → slow down or use cookies from a real logged-in browser session.

Example fix

null

// Probe the scrape target and treat non-2xx as a stop signal
const r = await fetch('https://weread.qq.com/web/search/books?keyword=test', { headers: { 'User-Agent': WEREAD_UA } });
if (!r.ok) console.warn(`search page unhealthy (HTTP ${r.status}); expect CommandExecutionError`);

try {
  const books = await runWereadSearch(query);
} catch (e) {
  const m = /HTTP (\d{3})/.exec(String(e.message));
  if (m && ['403', '429'].includes(m[1])) {
    await sleep(10000); // back off WAF/rate limit
    return retryWithFewerRequests();
  }
  if (m && m[1].startsWith('5')) return fallbackToApiOnly();
  throw e;
}

When it happens

Trigger: GET https://weread.qq.com/web/search/books?keyword=... returns 403 (bot/rate-limit protection), 429 (rate limited), 404 (route removed after a web app redesign), or 5xx (server outage). Any command invocation of `weread search` can hit this.

Common situations: WeRead WAF rejecting the spoofed Chrome User-Agent due to missing browser cookies/headers; hammering the search endpoint in a loop; WeRead renaming or removing the /web/search/books route in a frontend deploy; regional outages of weread.qq.com.

Related errors

• FETCH_ERROR
• 1point3acres request failed: HTTP ${res.status} ${res.status
• ${label} returned HTTP ${outcome.status}
• HTTP ${code}
• Search failed: HTTP ${res.status}

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/c03ec54b560f5922. Report an issue: GitHub.