{"record":{"id":"dbf0f067f646773c","repo":"jackwener/OpenCLI","slug":"failed-to-scrape-indeed-search-dom-e-message","errorCode":null,"errorMessage":"Failed to scrape Indeed search DOM: ${e?.message ?? e}","messagePattern":"Failed to scrape Indeed search DOM: (.+?)","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/indeed/search.js","lineNumber":94,"sourceCode":"                    const tags = Array.from(b.querySelectorAll('.metadataContainer li span'))\n                        .map(s => (s.textContent || '').trim())\n                        .filter(Boolean);\n                    out.push({\n                        jk,\n                        title: b.querySelector('h2.jobTitle span')?.textContent?.trim() ?? '',\n                        company: b.querySelector('[data-testid=\"company-name\"]')?.textContent?.trim() ?? '',\n                        location: b.querySelector('[data-testid=\"text-location\"]')?.textContent?.trim() ?? '',\n                        salary: b.querySelector('.salary-snippet-container span')?.textContent?.trim() ?? '',\n                        tags,\n                    });\n                }\n                const blockedHeadline = document.title || '';\n                const challenge = blockedHeadline.includes('Just a moment') || !!document.querySelector('[id^=\"cf-\"]');\n                return { cards: out, challenge, ready };\n            })()`);\n        }\n        catch (e) {\n            throw new CommandExecutionError(`Failed to scrape Indeed search DOM: ${e?.message ?? e}`, 'The page may not have fully loaded; try again.');\n        }\n\n        if (cards?.challenge) {\n            throw new CommandExecutionError('Indeed served a Cloudflare challenge page', 'Open https://www.indeed.com in the connected browser and clear the challenge, then retry.');\n        }\n        if (!cards?.ready) {\n            throw new CommandExecutionError('Indeed search page did not expose result or empty-state markers within 15s', 'Indeed may still be loading or the DOM shape may have changed; retry after opening Indeed in the connected browser.');\n        }\n\n        const list = Array.isArray(cards?.cards) ? cards.cards : [];\n        if (list.length === 0) {\n            throw new EmptyResultError('indeed search', `No Indeed jobs matched \"${query}\"${location ? ` in ${location}` : ''}`);\n        }\n        return list.slice(0, limit).map((c, i) => searchCardToRow(c, start + i + 1));\n    },\n});\n","sourceCodeStart":76,"sourceCodeEnd":111,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/indeed/search.js#L76-L111","documentation":"Wrapper error thrown when the Indeed search-page DOM scraping script throws inside page.evaluate. As with the job detail variant, it wraps the underlying message and usually means the page had not fully loaded when selectors were evaluated.","triggerScenarios":"The evaluate() call that collects search result cards and challenge flags throws during the indeed search command; e?.message is interpolated.","commonSituations":"Slow/partial page load causing selector access errors; navigation interrupted mid-evaluation; browser tab crashed; extension or CSP interference; Indeed served an unexpected page variant that broke the script.","solutions":["Retry the search to let the page fully load","Open the Indeed search URL in the connected browser, confirm rendering, then retry","Disable interfering browser extensions and retry","Increase wait time before evaluation (waitForSelector on result containers)","If persistent, check whether Indeed changed search DOM structure and update the scraper"],"exampleFix":"// before\nconst cards = await page.evaluate(main);\n// after\nawait page.waitForSelector('[data-testid=\"job-card\"] , .job_seen_beacon', { timeout: 20000 }).catch(() => {});\nconst cards = await page.evaluate(main);","handlingStrategy":"try-catch","validationCode":"await page.waitForSelector('.job_seen_beacon, [data-testid=\"job-card\"], .empty_serp', { timeout: 20000 }).catch(() => {});","typeGuard":null,"tryCatchPattern":"try {\n  const rows = await indeed.search({ query, location });\n} catch (e) {\n  if (/Failed to scrape Indeed search DOM/.test(e.message)) {\n    // back off and retry once with a fresh page\n  }\n  throw e;\n}","preventionTips":["Wait for search-result or empty-state selectors before evaluating","Retry transient evaluation failures with backoff","Keep the connected browser free of disruptive extensions","Update scraper selectors when Indeed changes markup"],"tags":["scraping","dom","page-load","indeed"],"backgroundTag":"scrape-dom-evaluation-failed","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}