jackwener/OpenCLI · error · CommandExecutionError

Indeed served a Cloudflare challenge page

Error message

Indeed served a Cloudflare challenge page

What it means

Thrown when the scraped job detail page is detected as a Cloudflare challenge: document.title contains 'Just a moment' or an element matching [id^="cf-"] is present. Indeed shields itself with Cloudflare and the CLI cannot (and will not) bypass it.

Source

Thrown at clis/indeed/job.js:66

                const notFound = !!document.querySelector('[data-testid="error-page"]') || /Page Not Found|not found/i.test(document.querySelector('h1')?.textContent || '');
                const title = document.querySelector('h1')?.textContent?.trim() ?? '';
                const company = document.querySelector('[data-testid="inlineHeader-companyName"] a, [data-testid="inlineHeader-companyName"], [data-company-name="true"]')?.textContent?.trim() ?? '';
                const location = document.querySelector('[data-testid="jobsearch-JobInfoHeader-companyLocation"] div, [data-testid="inlineHeader-companyLocation"]')?.textContent?.trim() ?? '';
                const salary = document.querySelector('[id*="salaryInfoAndJobType"] span, [data-testid="job-salary"]')?.textContent?.trim() ?? '';
                const jobType = Array.from(document.querySelectorAll('[id*="salaryInfoAndJobType"] span, [data-testid="job-type"]'))
                    .map(s => (s.textContent || '').trim())
                    .filter(t => t && t !== salary)
                    .join(', ');
                const description = document.querySelector('#jobDescriptionText')?.innerText?.trim() ?? '';
                return { ready, challenge, notFound, title, company, location, salary, jobType, description };
            })()`);
        }
        catch (e) {
            throw new CommandExecutionError(`Failed to scrape Indeed job detail DOM: ${e?.message ?? e}`, 'The page may not have fully loaded; try again.');
        }

        if (detail?.challenge) {
            throw new CommandExecutionError('Indeed served a Cloudflare challenge page', 'Open https://www.indeed.com in the connected browser and clear the challenge, then retry.');
        }
        if (!detail?.ready) {
            throw new CommandExecutionError('Indeed job page did not expose detail or error markers within 15s', 'Indeed may still be loading or the DOM shape may have changed; retry after opening Indeed in the connected browser.');
        }
        if (detail?.notFound || (!detail?.title && !detail?.description)) {
            throw new EmptyResultError('indeed job', `No Indeed job posting found for jk "${jk}"`);
        }

        return [{
            id: jk,
            title: detail.title.replace(/\s+/g, ' ').trim(),
            company: detail.company.replace(/\s+/g, ' ').trim(),
            location: detail.location.replace(/\s+/g, ' ').trim(),
            salary: detail.salary.replace(/\s+/g, ' ').trim(),
            job_type: detail.jobType.replace(/\s+/g, ' ').trim(),
            description: detail.description,
            url,
        }];

View on GitHub (pinned to 49907e53dc)

Solutions

  1. Open https://www.indeed.com in the connected browser and manually solve the challenge, then retry
  2. Use a normal (non-headless) browser session with a real user profile
  3. Switch network: disable VPN/proxy or use a residential IP
  4. Slow down request rate / add delays between Indeed requests
  5. Clear and re-establish a Cloudflare clearance by browsing Indeed normally once before scraping
Defensive patterns

Strategy: retry

Validate before calling

const title = await page.title();
if (title.includes('Just a moment') || (await page.$('[id^="cf-"]'))) {
  // solve the challenge manually in the connected browser before running the command
}

Try / catch

try {
  const job = await indeed.job(jk);
} catch (e) {
  if (e.message.includes('Cloudflare challenge')) {
    // prompt user to clear the challenge in the connected browser, wait, then retry
  }
  throw e;
}

Prevention

When it happens

Trigger: detail.challenge is true after evaluating the job page DOM, i.e. headline includes 'Just a moment' or a cf-* element exists.

Common situations: Scraping from a datacenter/VPN IP flagged by Cloudflare; headless or automation-flagged browser profile; too many rapid requests from the same session; cookies cleared so no clearance token exists.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/25308b295eb4b6dc. Report an issue: GitHub.