jackwener/OpenCLI · error · Error

Timed out waiting for selector "${kwargs['wait-for']}" in ma

Error message

Timed out waiting for selector "${kwargs['wait-for']}" in main document or same-origin iframes

What it means

clis/web/read.js waits for a CSS selector to appear before extracting page content. It injects a script (buildWaitForSelectorAcrossFramesJs) that polls the main document and same-origin iframes for up to --wait seconds; if the selector is never found it throws this timeout error so the caller knows the page did not reach the expected state.

Source

Thrown at clis/web/read.js:432

    func: async (page, kwargs, debug = false) => {
        const url = kwargs.url;
        const waitSeconds = kwargs.wait ?? 3;
        const waitUntil = normalizeWaitUntil(kwargs['wait-until']);
        const frameMode = normalizeFrameMode(kwargs.frames);
        const shouldDiagnose = boolish(kwargs.diagnose) || debug || !!process.env.OPENCLI_VERBOSE;
        const networkEntries = [];
        const captureSupported = (waitUntil === 'networkidle' || shouldDiagnose)
            ? await maybeStartNetworkCapture(page)
            : false;
        // Navigate to the target URL
        await page.goto(url);
        if (kwargs['wait-for']) {
            const waitResult = await page.evaluate(buildWaitForSelectorAcrossFramesJs(String(kwargs['wait-for']), waitSeconds * 1000));
            if (waitResult?.invalidSelector) {
                throw new Error(`Invalid --wait-for selector "${kwargs['wait-for']}": ${waitResult.error || 'querySelector failed'}`);
            }
            if (!waitResult?.ok) {
                throw new Error(`Timed out waiting for selector "${kwargs['wait-for']}" in main document or same-origin iframes`);
            }
        } else if (waitUntil !== 'networkidle') {
            await page.wait(waitSeconds);
        }
        if (waitUntil === 'networkidle') {
            if (!captureSupported) {
                throw new Error('Network capture is unavailable, so --wait-until networkidle cannot be satisfied');
            }
            const idle = await waitForNetworkIdle(page, waitSeconds, networkEntries);
            if (!idle?.ok) {
                throw new Error(`Timed out waiting for network idle after ${waitSeconds}s`);
            }
        }
        // Extract article content using browser-side heuristics
        const data = await page.evaluate(buildRenderAwareExtractorJs({ frames: frameMode }));
        if (captureSupported) await drainNetworkCapture(page, networkEntries);
        if (shouldDiagnose) process.stderr.write(formatDiagnostics(data, networkEntries, captureSupported));
        // Determine Referer from URL for image downloads

View on GitHub (pinned to 49907e53dc)

Solutions

Verify the selector matches in the browser devtools of the loaded page (not a different variant/redirect).
Increase the wait timeout (e.g. --wait 15) or switch --wait-until to networkidle.
If the element is in a cross-origin iframe, run the read against the iframe URL directly instead.
Check whether the element only appears after interaction; use a selector for a container that exists earlier instead.
Confirm the page is not being redirected to a login/captcha state.

Example fix

// before
web read https://spa.example.com --wait-for '#results .item' --wait 3
// after
web read https://spa.example.com --wait-for '#results' --wait 15 --wait-until networkidle

Defensive patterns

Strategy: validation

Validate before calling

// Pre-check selector presence before invoking with a hard timeout:
const exists = await page.evaluate(`!!document.querySelector('${sel.replace(/'/g, "\\'")}')`);
if (!exists) console.warn(`selector ${sel} not present yet — increase --wait`);

Type guard

function isWaitResult(r) { return r != null && typeof r === 'object' && 'ok' in r; }

Try / catch

try {
  await readPage(url, { waitFor: sel, waitSeconds: 15 });
} catch (e) {
  if (String(e.message).includes('Timed out waiting for selector')) {
    // fall back to longer wait or extract without the selector
    await readPage(url, { waitUntil: 'networkidle', waitSeconds: 30 });
  } else throw e;
}

Prevention

Validate selectors in devtools against the actual (post-redirect) page
Prefer --wait-for on an early stable container over late-rendered leaf nodes
Set generous --wait values for SPAs and slow networks
Remember cross-origin iframes are invisible to the frame-aware poller

When it happens

Trigger: Running the web read command with --wait-for <selector> when the element does not appear within the wait window: wrong selector syntax (already caught earlier as invalidSelector), element rendered only after user interaction, element inside a cross-origin iframe the poller cannot see, lazy-loaded content slower than the timeout, or a page that failed to load the node at all.

Common situations: Scraping SPA pages that hydrate slowly; targeting elements behind a cookie/consent banner; selectors written for a desktop page but hitting a mobile redirect; content in a third-party (cross-origin) embed; setting --wait too low on a slow network.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/4c04438f304d5dcf. Report an issue: GitHub.