{"record":{"id":"b36715e3ee0b87f7","repo":"jackwener/OpenCLI","slug":"extraction-drift-b36715","errorCode":"extraction_drift","errorMessage":"[taxonomy=extraction_drift] site=${site} command=detail detail extraction returned invalid payload: ${targetUrl}","messagePattern":"\\[taxonomy=extraction_drift\\] site=(.+?) command=detail detail extraction returned invalid payload: (.+?)","errorType":"error_code","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"clis/jianyu/shared/procurement-detail.js","lineNumber":61,"sourceCode":"      };\n    })()\n  `);\n}\nexport async function runProcurementDetail(page, { url, site, query = '', }) {\n    const targetUrl = cleanText(url);\n    if (!targetUrl) {\n        throw taxonomyError('relay_unavailable', {\n            site,\n            command: 'detail',\n            detail: 'missing required detail url',\n        });\n    }\n    let lastError = null;\n    for (let attempt = 1; attempt <= DETAIL_MAX_ATTEMPTS; attempt += 1) {\n        try {\n            const payload = await extractDetailPayload(page, targetUrl);\n            if (!payload || typeof payload !== 'object') {\n                throw taxonomyError('extraction_drift', {\n                    site,\n                    command: 'detail',\n                    detail: `detail extraction returned invalid payload: ${targetUrl}`,\n                });\n            }\n            const row = payload;\n            const title = cleanText(row.title);\n            const detailText = cleanText(row.detailText);\n            const publishTime = cleanText(row.publishTime);\n            const authGateText = cleanText(`${title} ${detailText}`);\n            if (DETAIL_AUTH_CHALLENGE_PATTERNS.some((pattern) => pattern.test(authGateText))) {\n                throw taxonomyError('selector_drift', {\n                    site,\n                    command: 'detail',\n                    detail: `detail page blocked by verification challenge: ${targetUrl}`,\n                });\n            }\n            if (!title && !detailText) {","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/jianyu/shared/procurement-detail.js#L43-L79","documentation":"Inside runProcurementDetail's retry loop, extractDetailPayload must return a non-null object payload. If the extraction returns null, a primitive, or otherwise invalid value, the code throws a taxonomy error with code 'extraction_drift' naming the target URL — meaning the page structure no longer matches the extractor's expectations.","triggerScenarios":"extractDetailPayload returns null/non-object for a targetUrl, typically after DETAIL_MAX_ATTEMPTS-independent immediate failure on every attempt of the loop (last attempt's error propagates).","commonSituations":"Detail page markup changed so the extractor returns null; the page served a soft-404 or redirect; JS-rendered content not ready when extraction runs; anti-bot interstitial replacing the article body.","solutions":["Fetch the targetUrl in a normal browser and compare its DOM against the extractor's selectors","Update extractDetailPayload selectors for the new page structure","Add a longer settle wait for JS-rendered content before extraction","Check for redirects/soft-404s; skip or flag such URLs upstream"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// confirm the page actually rendered content before extraction\nawait page.waitForSelector('[data-detail-body], .article-content', { timeout: 15000 }).catch(() => {});","typeGuard":"function isDetailDrift(e) {\n  return e instanceof Error && /extraction_drift.*detail extraction/.test(e.message);\n}","tryCatchPattern":"try {\n  return await runProcurementDetail(page, { site, url });\n} catch (e) {\n  if (isDetailDrift(e)) {\n    await sleep(3000); // allow JS render, retry once\n    return await runProcurementDetail(page, { site, url });\n  }\n  throw e;\n}","preventionTips":["Wait for network idle / key selectors before extraction","Snapshot failing detail pages for selector updates","Keep extractDetailPayload selectors current with site releases","Alert per-site so drift is attributed quickly"],"tags":["scraping","extraction-drift","dom"],"backgroundTag":"extraction-drift","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}