{"record":{"id":"f658ae2e3e010ce0","repo":"we-promise/sure","slug":"yahoo-finance-authentication-failed-after-crumb-re","errorCode":null,"errorMessage":"Yahoo Finance authentication failed after crumb refresh","messagePattern":"Yahoo Finance authentication failed after crumb refresh","errorType":"exception","errorClass":"Provider::YahooFinance::AuthenticationError","httpStatus":null,"severity":"critical","filePath":"app/models/provider/yahoo_finance.rb","lineNumber":259,"sourceCode":"      response = authenticated_client(cookie).get(\"#{base_url}/v10/finance/quoteSummary/#{symbol}\") do |req|\n        req.params[\"modules\"] = \"assetProfile,price,quoteType\"\n        req.params[\"crumb\"] = crumb\n      end\n\n      data = JSON.parse(response.body)\n\n      # Check for auth errors in response body\n      if data.dig(\"quoteSummary\", \"error\", \"code\") == \"Unauthorized\"\n        # Clear cached crumb and retry once\n        clear_crumb_cache\n        cookie, crumb = fetch_cookie_and_crumb\n        response = authenticated_client(cookie).get(\"#{base_url}/v10/finance/quoteSummary/#{symbol}\") do |req|\n          req.params[\"modules\"] = \"assetProfile,price,quoteType\"\n          req.params[\"crumb\"] = crumb\n        end\n        data = JSON.parse(response.body)\n        if data.dig(\"quoteSummary\", \"error\", \"code\") == \"Unauthorized\"\n          raise AuthenticationError, \"Yahoo Finance authentication failed after crumb refresh\"\n        end\n      end\n\n      result = data.dig(\"quoteSummary\", \"result\", 0)\n\n      raise Error, \"No security info found for #{symbol}\" unless result\n\n      asset_profile = result[\"assetProfile\"] || {}\n      price_info = result[\"price\"] || {}\n      quote_type = result[\"quoteType\"] || {}\n\n      security_info = SecurityInfo.new(\n        symbol: symbol,\n        name: price_info[\"longName\"] || price_info[\"shortName\"] || quote_type[\"longName\"] || quote_type[\"shortName\"],\n        links: asset_profile[\"website\"],\n        logo_url: nil, # Yahoo doesn't provide reliable logo URLs\n        description: asset_profile[\"longBusinessSummary\"],\n        kind: map_security_type(quote_type[\"quoteType\"]),","sourceCodeStart":241,"sourceCodeEnd":277,"githubUrl":"https://github.com/we-promise/sure/blob/e69894adb92547273377398c15f45c979cd9416a/app/models/provider/yahoo_finance.rb#L241-L277","documentation":"Provider::YahooFinance::AuthenticationError raised in fetch_security_info when the quoteSummary endpoint answers 'Unauthorized' even after the client cleared its cached cookie/crumb pair and fetched a fresh one. Yahoo's unofficial API requires a session cookie plus a crumb token; two consecutive Unauthorized responses mean the cookie/crumb flow itself is broken (blocked, expired, or rejected).","triggerScenarios":"The /v10/finance/quoteSummary call returning error.code 'Unauthorized' with a stale crumb, then again after clear_crumb_cache + fetch_cookie_and_crumb — e.g. the consent page served instead of a cookie, the crumb endpoint returning 'too many requests' crumbs (see INVALID_CRUMBS), or Yahoo blocking the client's IP/User-Agent from crumb issuance.","commonSituations":"Yahoo tightening anti-bot measures so cookie+crumb acquisition silently fails; datacenter IPs flagged; the cached cookie store (shared via Rails.cache) poisoned with a cookie another blocked process obtained; outdated User-Agent pool after a Yahoo crackdown (the codebase updates USER_AGENTS periodically for exactly this).","solutions":["Wait and retry later — Yahoo blocks are usually temporary; check provider.health_status for the tracked rate_limited/unavailable state","Test fetch_cookie_and_crumb in isolation (console) to see whether crumb issuance returns a valid token or an error/HTML","Rotate the egress IP or update the USER_AGENTS pool to current browser versions when Yahoo starts rejecting old fingerprints","Clear the yahoo_finance cookie/crumb cache keys if a poisoned cookie is cached; fall back to another securities provider for info lookups"],"exampleFix":"// before\ninfo = provider.fetch_security_info(symbol: \"AAPL\", exchange_operating_mic: \"XNAS\")\n\n// after\nbegin\n  info = provider.fetch_security_info(symbol: \"AAPL\", exchange_operating_mic: \"XNAS\")\nrescue Provider::YahooFinance::AuthenticationError\n  if provider.health_status == :rate_limited\n    RetryableSyncJob.perform_later(wait: 30.minutes)\n  else\n    info = fallback_provider.fetch_security_info(symbol: \"AAPL\", exchange_operating_mic: \"XNAS\")\n  end\nend","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"begin\n  provider.fetch_security_info(symbol:, exchange_operating_mic:)\nrescue Provider::YahooFinance::AuthenticationError\n  if provider.health_status == :rate_limited\n    RetryableJob.perform_later(wait: 30.minutes)\n  else\n    fallback_provider.fetch_security_info(symbol:, exchange_operating_mic:)\n  end\nend","preventionTips":["Check provider.health_status before hitting crumb-authenticated endpoints; it already tracks Yahoo blocks","Keep the USER_AGENTS pool current (the codebase refreshes it when Yahoo tightens checks)","Cache successful quoteSummary lookups to minimize crumb-protected calls","Clear poisoned cookie/crumb cache entries when blocks are suspected"],"tags":["ruby","yahoo-finance","authentication","cookie-crumb","bot-detection"],"backgroundTag":"cookie-crumb-auth-failed","analyzedSha":"e69894adb92547273377398c15f45c979cd9416a","analyzedAt":"2026-08-21T18:22:41.165Z","schemaVersion":2},"datasetVersion":"2026-08-21T23:17:16.201Z"}