{"record":{"id":"caaa3c7bf217d3b7","repo":"can1357/oh-my-pi","slug":"google-blocked-the-browser-search-with-an-automate","errorCode":null,"errorMessage":"Google blocked the browser search with an automated-traffic challenge. Try another web search provider or retry later.","messagePattern":"Google blocked the browser search with an automated-traffic challenge\\. Try another web search provider or retry later\\.","errorType":"exception","errorClass":"SearchProviderError","httpStatus":429,"severity":"error","filePath":"packages/coding-agent/src/web/search/providers/google.ts","lineNumber":146,"sourceCode":"\t\t\treferer: GOOGLE_HOME_URL,\n\t\t\tbrowser: {\n\t\t\t\thomeUrl: GOOGLE_HOME_URL,\n\t\t\t\tready: { selector: \"a h3\", timeoutMs: RESULT_RENDER_TIMEOUT_MS },\n\t\t\t\tshouldFallback: candidate => blockReason(candidate) !== undefined,\n\t\t\t},\n\t\t});\n\t} catch (error) {\n\t\tif (error instanceof SearchProviderError || params.signal?.aborted) throw error;\n\t\tif (signal.aborted) {\n\t\t\tthrow new SearchProviderError(\"google\", \"Google browser search timed out.\", 504);\n\t\t}\n\t\tconst message = error instanceof Error ? error.message : String(error);\n\t\tthrow new SearchProviderError(\"google\", `Google browser search failed: ${message}`, 503);\n\t}\n\n\tconst blocked = blockReason(page);\n\tif (blocked === \"traffic\") {\n\t\tthrow new SearchProviderError(\n\t\t\t\"google\",\n\t\t\t\"Google blocked the browser search with an automated-traffic challenge. Try another web search provider or retry later.\",\n\t\t\t429,\n\t\t);\n\t}\n\tif (page.status < 200 || page.status >= 300) {\n\t\tthrow new SearchProviderError(\"google\", `Google HTML error (${page.status})`, page.status);\n\t}\n\tif (blocked === \"javascript\") {\n\t\tthrow new SearchProviderError(\n\t\t\t\"google\",\n\t\t\t\"Google returned its JavaScript challenge instead of rendered search results.\",\n\t\t\t429,\n\t\t);\n\t}\n\treturn page.html;\n}\n","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/packages/coding-agent/src/web/search/providers/google.ts#L128-L164","documentation":"Thrown by callGoogleHtml after the page loads when blockReason() classifies the response as an automated-traffic challenge — status 403/429, a /sorry/ redirect URL, or HTML containing 'unusual traffic' / g-recaptcha markers. Status 429 signals rate limiting/abuse detection by Google against the scraping approach.","triggerScenarios":"GET of the Google Search results URL returns status 403 or 429, redirects to a google.com/sorry/ page, or its HTML matches /unusual traffic|detected unusual traffic|g-recaptcha/i (including via the browser fallback's shouldFallback probe).","commonSituations":"High search volume from one IP (shared CI runners, corporate NAT, VPN endpoints); datacenter IPs Google distrusts; no cookies/captcha tokens available for the headless browser.","solutions":["Retry later, ideally with backoff; the block is usually time-limited.","Configure a different web search provider (Jina, Gemini, Exa, etc.) which uses APIs rather than scraping.","Change network egress (residential IP, different VPN/exit node) if the IP is flagged.","Reduce search frequency/concurrency to stay under Google's abuse thresholds."],"exampleFix":"// before: tight loop against google scraping\nfor (const q of queries) await search({ provider: \"google\", query: q });\n// after: fall back to an API-backed provider\nconst provider = pickProvider(); // e.g. \"jina\" when google is blocked\nfor (const q of queries) await search({ provider, query: q });","handlingStrategy":"fallback","validationCode":null,"typeGuard":"function isTrafficBlocked(e: unknown): boolean {\n  return e instanceof SearchProviderError && e.provider === 'google' && e.statusCode === 429;\n}","tryCatchPattern":"try {\n  return await search({ provider: 'google', query });\n} catch (err) {\n  if (isTrafficBlocked(err)) return await search({ provider: 'jina', query }); // API-backed fallback\n  throw err;\n}","preventionTips":["Throttle and queue Google scraping searches; avoid parallel bursts.","Avoid datacenter/VPN egress IPs that Google flags.","Treat 429 as non-retryable short-term; back off minutes, not seconds.","Prefer API-backed providers for production workloads."],"tags":["rate-limit","captcha","google","scraping","search-provider"],"backgroundTag":"blocked-by-captcha","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}