jackwener/OpenCLI · error · Error

extraction_drift

extraction_drift

Error message

[taxonomy=extraction_drift] site=${site} command=detail detail extraction returned invalid payload: ${targetUrl}

What it means

Inside runProcurementDetail's retry loop, extractDetailPayload must return a non-null object payload. If the extraction returns null, a primitive, or otherwise invalid value, the code throws a taxonomy error with code 'extraction_drift' naming the target URL — meaning the page structure no longer matches the extractor's expectations.

Source

Thrown at clis/jianyu/shared/procurement-detail.js:61

      };
    })()
  `);
}
export async function runProcurementDetail(page, { url, site, query = '', }) {
    const targetUrl = cleanText(url);
    if (!targetUrl) {
        throw taxonomyError('relay_unavailable', {
            site,
            command: 'detail',
            detail: 'missing required detail url',
        });
    }
    let lastError = null;
    for (let attempt = 1; attempt <= DETAIL_MAX_ATTEMPTS; attempt += 1) {
        try {
            const payload = await extractDetailPayload(page, targetUrl);
            if (!payload || typeof payload !== 'object') {
                throw taxonomyError('extraction_drift', {
                    site,
                    command: 'detail',
                    detail: `detail extraction returned invalid payload: ${targetUrl}`,
                });
            }
            const row = payload;
            const title = cleanText(row.title);
            const detailText = cleanText(row.detailText);
            const publishTime = cleanText(row.publishTime);
            const authGateText = cleanText(`${title} ${detailText}`);
            if (DETAIL_AUTH_CHALLENGE_PATTERNS.some((pattern) => pattern.test(authGateText))) {
                throw taxonomyError('selector_drift', {
                    site,
                    command: 'detail',
                    detail: `detail page blocked by verification challenge: ${targetUrl}`,
                });
            }
            if (!title && !detailText) {

View on GitHub (pinned to 49907e53dc)

Solutions

1. Fetch the targetUrl in a normal browser and compare its DOM against the extractor's selectors
2. Update extractDetailPayload selectors for the new page structure
3. Add a longer settle wait for JS-rendered content before extraction
4. Check for redirects/soft-404s; skip or flag such URLs upstream

// confirm the page actually rendered content before extraction
await page.waitForSelector('[data-detail-body], .article-content', { timeout: 15000 }).catch(() => {});

function isDetailDrift(e) {
  return e instanceof Error && /extraction_drift.*detail extraction/.test(e.message);
}

try {
  return await runProcurementDetail(page, { site, url });
} catch (e) {
  if (isDetailDrift(e)) {
    await sleep(3000); // allow JS render, retry once
    return await runProcurementDetail(page, { site, url });
  }
  throw e;
}

When it happens

Trigger: extractDetailPayload returns null/non-object for a targetUrl, typically after DETAIL_MAX_ATTEMPTS-independent immediate failure on every attempt of the loop (last attempt's error propagates).

Common situations: Detail page markup changed so the extractor returns null; the page served a soft-404 or redirect; JS-rendered content not ready when extraction runs; anti-bot interstitial replacing the article body.

Related errors

• Failed to extract Booking.com cards: ${err?.message || err}
• No visible Claude messages were found for conversation ${id}
• No Claude conversation history was visible on /recents.
• No visible Claude messages were found in the current convers
• coupang search extraction failed: ${error?.message || error}

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/b36715e3ee0b87f7. Report an issue: GitHub.