we-promise/sure · error · Family::DataImporter::InvalidRecordError

invalid_import_record

invalid_import_record

Error message

#{record_type} has invalid #{field}: #{value.inspect}

What it means

Provider::YahooFinance::AuthenticationError raised in fetch_authenticated_chart when Yahoo returns chart.error.code == 'Unauthorized' twice: once initially, and again after the provider cleared its crumb cache, fetched a fresh cookie+crumb, and retried the same chart request. Yahoo's v8 endpoints require a valid A1/A3 cookie plus a crumb query parameter; when both attempts are rejected, the credentials context (cookie) itself is being refused, not just a stale crumb.

Source

Thrown at app/models/family/data_importer.rb:242

      increment_summary(record_type, :skipped)
      nil
    end

    def require_source_id!(record_type, source_id)
      return if source_id.present? || !@strict_references

      increment_summary(record_type, :failed)
      raise MissingReferenceError.new(
        record_type: record_type,
        source_type: record_type,
        source_id: "(blank)"
      )
    end

    def invalid_record!(record_type, field, value)
      if @strict_references
        increment_summary(record_type, :failed)
        raise InvalidRecordError.new(record_type: record_type, field: field, value: value)
      end

      increment_summary(record_type, :skipped)
      nil
    end

    def session_entry_source
      return unless @import_session

      "sure_import_session:#{@import_session.id}"
    end

    def session_entry_external_id(record_type, source_id)
      return if @import_session.blank? || source_id.blank?

      "#{record_type}:#{source_id}"
    end

View on GitHub (pinned to e69894adb9)

Solutions

1. Back off and retry later — most double-unauthorized failures are IP-reputation rate limiting
2. Clear the Rails.cache crumb entries (clear_crumb_cache) so a fresh cookie/crumb pair is fetched on next call
3. Reduce request frequency (throttle_request settings) to stay under Yahoo's tolerance
4. Route via a different egress IP or residential proxy for server deployments
5. If persistent, switch the affected lookups to a configured alternative provider (Tiingo/EODHD)

Example fix

# before
prices = provider.fetch_security_prices(symbol: s, start_date: a, end_date: b)

# after
begin
  prices = provider.fetch_security_prices(symbol: s, start_date: a, end_date: b)
rescue Provider::YahooFinance::AuthenticationError => e
  Rails.cache.delete_matched("yahoo_crumb*") if e.message.include?("crumb refresh")
  sleep 60
  retry if (attempts += 1) < 3
  raise
end

attempts = 0
begin
  attempts += 1
  data = provider.fetch_security_prices(symbol: s, start_date: a, end_date: b)
rescue Provider::YahooFinance::AuthenticationError => e
  raise unless e.message.include?("crumb refresh") && attempts < 3
  sleep(60 * attempts) # back off; IP-reputation limiting usually clears
  retry
end

When it happens

Trigger: Repeated chart fetches from a rate-limited or flagged IP where Yahoo invalidates cookies immediately; Yahoo-side A/B changes to the consent/crumb flow making freshly fetched cookies unusable; clock-skewed requests invalidating the session; the fc.yahoo.com cookie handoff returning a consent-redirect cookie instead of the auth cookie.

Common situations: Bulk sync jobs hammering Yahoo from a datacenter IP; after Yahoo changes its anti-bot flow (historically frequent); local dev behind VPN egress shared with abusers; running many provider instances with a shared cache serving poisoned crumbs.

Related errors

• Failed to obtain Yahoo Finance crumb
• Mercury provider is not configured
• Could not save that passkey or security key. Please try agai
• Lunchflow provider is not configured
• Yahoo Finance rate limit exceeded

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/ab340de1c0faba89. Report an issue: GitHub.