{"record":{"id":"4c04438f304d5dcf","repo":"jackwener/OpenCLI","slug":"timed-out-waiting-for-selector-kwargs-wait-for","errorCode":null,"errorMessage":"Timed out waiting for selector \"${kwargs['wait-for']}\" in main document or same-origin iframes","messagePattern":"Timed out waiting for selector \"(.+?)\" in main document or same-origin iframes","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"clis/web/read.js","lineNumber":432,"sourceCode":"    func: async (page, kwargs, debug = false) => {\n        const url = kwargs.url;\n        const waitSeconds = kwargs.wait ?? 3;\n        const waitUntil = normalizeWaitUntil(kwargs['wait-until']);\n        const frameMode = normalizeFrameMode(kwargs.frames);\n        const shouldDiagnose = boolish(kwargs.diagnose) || debug || !!process.env.OPENCLI_VERBOSE;\n        const networkEntries = [];\n        const captureSupported = (waitUntil === 'networkidle' || shouldDiagnose)\n            ? await maybeStartNetworkCapture(page)\n            : false;\n        // Navigate to the target URL\n        await page.goto(url);\n        if (kwargs['wait-for']) {\n            const waitResult = await page.evaluate(buildWaitForSelectorAcrossFramesJs(String(kwargs['wait-for']), waitSeconds * 1000));\n            if (waitResult?.invalidSelector) {\n                throw new Error(`Invalid --wait-for selector \"${kwargs['wait-for']}\": ${waitResult.error || 'querySelector failed'}`);\n            }\n            if (!waitResult?.ok) {\n                throw new Error(`Timed out waiting for selector \"${kwargs['wait-for']}\" in main document or same-origin iframes`);\n            }\n        } else if (waitUntil !== 'networkidle') {\n            await page.wait(waitSeconds);\n        }\n        if (waitUntil === 'networkidle') {\n            if (!captureSupported) {\n                throw new Error('Network capture is unavailable, so --wait-until networkidle cannot be satisfied');\n            }\n            const idle = await waitForNetworkIdle(page, waitSeconds, networkEntries);\n            if (!idle?.ok) {\n                throw new Error(`Timed out waiting for network idle after ${waitSeconds}s`);\n            }\n        }\n        // Extract article content using browser-side heuristics\n        const data = await page.evaluate(buildRenderAwareExtractorJs({ frames: frameMode }));\n        if (captureSupported) await drainNetworkCapture(page, networkEntries);\n        if (shouldDiagnose) process.stderr.write(formatDiagnostics(data, networkEntries, captureSupported));\n        // Determine Referer from URL for image downloads","sourceCodeStart":414,"sourceCodeEnd":450,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/web/read.js#L414-L450","documentation":"clis/web/read.js waits for a CSS selector to appear before extracting page content. It injects a script (buildWaitForSelectorAcrossFramesJs) that polls the main document and same-origin iframes for up to --wait seconds; if the selector is never found it throws this timeout error so the caller knows the page did not reach the expected state.","triggerScenarios":"Running the web read command with --wait-for <selector> when the element does not appear within the wait window: wrong selector syntax (already caught earlier as invalidSelector), element rendered only after user interaction, element inside a cross-origin iframe the poller cannot see, lazy-loaded content slower than the timeout, or a page that failed to load the node at all.","commonSituations":"Scraping SPA pages that hydrate slowly; targeting elements behind a cookie/consent banner; selectors written for a desktop page but hitting a mobile redirect; content in a third-party (cross-origin) embed; setting --wait too low on a slow network.","solutions":["Verify the selector matches in the browser devtools of the loaded page (not a different variant/redirect).","Increase the wait timeout (e.g. --wait 15) or switch --wait-until to networkidle.","If the element is in a cross-origin iframe, run the read against the iframe URL directly instead.","Check whether the element only appears after interaction; use a selector for a container that exists earlier instead.","Confirm the page is not being redirected to a login/captcha state."],"exampleFix":"// before\nweb read https://spa.example.com --wait-for '#results .item' --wait 3\n// after\nweb read https://spa.example.com --wait-for '#results' --wait 15 --wait-until networkidle","handlingStrategy":"validation","validationCode":"// Pre-check selector presence before invoking with a hard timeout:\nconst exists = await page.evaluate(`!!document.querySelector('${sel.replace(/'/g, \"\\\\'\")}')`);\nif (!exists) console.warn(`selector ${sel} not present yet — increase --wait`);","typeGuard":"function isWaitResult(r) { return r != null && typeof r === 'object' && 'ok' in r; }","tryCatchPattern":"try {\n  await readPage(url, { waitFor: sel, waitSeconds: 15 });\n} catch (e) {\n  if (String(e.message).includes('Timed out waiting for selector')) {\n    // fall back to longer wait or extract without the selector\n    await readPage(url, { waitUntil: 'networkidle', waitSeconds: 30 });\n  } else throw e;\n}","preventionTips":["Validate selectors in devtools against the actual (post-redirect) page","Prefer --wait-for on an early stable container over late-rendered leaf nodes","Set generous --wait values for SPAs and slow networks","Remember cross-origin iframes are invisible to the frame-aware poller"],"tags":["browser","timeout","selector","scraping"],"backgroundTag":"selector-timeout","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}