{"record":{"id":"368f8d00e752de18","repo":"jackwener/OpenCLI","slug":"extraction-drift","errorCode":"extraction_drift","errorMessage":"[taxonomy=extraction_drift] site=${site} command=search all rows rejected by quality gate (raw=${raw count})","messagePattern":"\\[taxonomy=extraction_drift\\] site=(.+?) command=search all rows rejected by quality gate \\(raw=(.+?)\\)","errorType":"error_code","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"clis/jianyu/shared/procurement-contract.js","lineNumber":291,"sourceCode":"    return deduped;\n}\nexport function formatTaxonomyError(taxonomy, { site, command, detail, }) {\n    return `[taxonomy=${taxonomy}] site=${site} command=${command} ${cleanText(detail)}`;\n}\nexport function taxonomyError(taxonomy, context) {\n    return new Error(formatTaxonomyError(taxonomy, context));\n}\nexport function toProcurementSearchRecords(rows, { site, query, limit, }) {\n    const normalizedRows = dedupeByTitleUrl(rows.map((row) => normalizeCoreRecord(row, { sourceSite: site })));\n    const accepted = [];\n    for (const row of normalizedRows) {\n        const rejectReason = qualityRejectReason(row, query);\n        if (rejectReason)\n            continue;\n        accepted.push(row);\n    }\n    if (normalizedRows.length > 0 && accepted.length === 0) {\n        throw taxonomyError('extraction_drift', {\n            site,\n            command: 'search',\n            detail: `all rows rejected by quality gate (raw=${normalizedRows.length})`,\n        });\n    }\n    return accepted\n        .slice(0, Math.max(1, limit))\n        .map((row, index) => ({\n        rank: index + 1,\n        ...row,\n    }));\n}\nexport function toProcurementDetailRecord({ title, url, contextText, publishTime, }, { site, query = '', }) {\n    const core = normalizeCoreRecord({\n        title,\n        url,\n        date: publishTime,\n        contextText,","sourceCodeStart":273,"sourceCodeEnd":309,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/jianyu/shared/procurement-contract.js#L273-L309","documentation":"toProcurementSearchRecords applies a per-site quality gate (qualityRejectReason) to each raw search row. If raw rows exist but every one is rejected, the library assumes the site's markup changed and parsers are extracting wrong data, so it throws a taxonomy error with code 'extraction_drift' rather than returning an empty-but-misleading result set.","triggerScenarios":"Calling the jianyu procurement 'search' command for a site where the parser returns rows but none pass qualityRejectReason (e.g. missing/malformed titles, wrong fields, filtered-out noise rows).","commonSituations":"Site redesign or markup changes breaking field extraction; quality-gate thresholds tightened while the site's data format shifted; scraping a site variant whose layout differs from the supported template; proxy/mirror serving stale or rewritten HTML.","solutions":["Dump the raw rows and check why each fails qualityRejectReason, then fix the parser/normalizer","Update selectors/regexes for the drifted site layout","Relax the quality gate only if the rows are genuinely valid","Pin/upgrade the library version that supports the current site markup"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"function rowsLookSane(rows) {\n  return rows.length === 0 || rows.some(r => r.title && r.title.length > 3 && r.url);\n}\nif (!rowsLookSane(rawRows)) console.warn('quality gate may reject all rows for site', site);","typeGuard":"function isExtractionDrift(e) {\n  return e instanceof Error && /\\[taxonomy=extraction_drift\\]/.test(e.message);\n}","tryCatchPattern":"try {\n  rows = await procurementSearch(page, { site, query });\n} catch (e) {\n  if (isExtractionDrift(e)) {\n    // all rows rejected: flag site for parser review, do not treat as empty results\n    reportDrift(site, 'search');\n    return [];\n  }\n  throw e;\n}","preventionTips":["Log raw rows alongside quality-gate rejections to debug drift fast","Alert on extraction_drift per site — it signals markup changes","Keep parser selectors per-site and versioned","Sample real pages in CI to catch drift before production"],"tags":["scraping","data-quality","extraction-drift"],"backgroundTag":"extraction-drift","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}