we-promise/sure · warning · Provider::Tiingo::RateLimitError

Tiingo hourly request limit reached

Error message

Tiingo hourly request limit reached (#{new_count}/#{max_requests_per_hour})

What it means

This is a client-side quota gate, not a server error. throttle_request first applies the interval throttle from RateLimitable, then atomically increments a Redis counter keyed 'tiingo:requests:<epoch-hour>' (expires in 7200s). When the returned count is >= max_requests_per_hour (ENV TIINGO_MAX_REQUESTS_PER_HOUR, default MAX_REQUESTS_PER_HOUR = 1000), it raises RateLimitError. The raise happens BEFORE the HTTP request, so no API call is wasted and Tiingo never saw the blocked request.

Solutions

Wait for the hour window to roll over -- the key is 'tiingo:requests:#{Time.current.to_i / 3600}', so the counter resets at the next epoch-hour boundary
If your Tiingo plan allows more, raise TIINGO_MAX_REQUESTS_PER_HOUR in the environment
Reduce request volume: widen cache TTLs, batch date ranges into single fetch_security_prices calls instead of per-date fetch_security_price, reuse search currency caching

Example fix

# before (per-date lookups, one request each)
dates.each { |d| provider.fetch_security_price(symbol: sym, date: d) }

# after (one ranged request)
provider.fetch_security_prices(symbol: sym, start_date: dates.first, end_date: dates.last)

Defensive patterns

Strategy: retry

Validate before calling

# Check the shared counter before issuing a request
count = Rails.cache.read("tiingo:requests:#{Time.current.to_i / 3600}").to_i
raise Provider::Tiingo::RateLimitError, 'Local hourly budget spent' if count >= ENV.fetch('TIINGO_MAX_REQUESTS_PER_HOUR', 1000).to_i

Try / catch

begin
  provider.fetch_security_prices(...)
rescue Provider::Tiingo::RateLimitError
  SyncRetryJob.set(wait: until_next_epoch_hour).perform_later(...) # window resets on the hour
end

Prevention

Batch date ranges into single ranged calls instead of per-date requests
Set TIINGO_MAX_REQUESTS_PER_HOUR to the real plan limit, not the default
Prefer cached currency/search data to avoid repeat search calls

When it happens

Trigger: Any Tiingo call (search_securities, fetch_security_prices, fetch_security_price, healthy?) once the whole app (all processes share the Redis counter) has made ~1000 Tiingo requests within the current wall-clock hour. The message includes the live count over the cap, e.g. '(1000/1000)'.

Common situations: Bulk price-refresh jobs sweeping large portfolios; a retry storm elsewhere re-requesting prices; default 1000/hr cap left in place while the account is on a paid tier that allows more; cache misses forcing repeated search calls.

Related errors

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/e56db25806a98d0c. Report an issue: GitHub.

Appendix: source

Thrown at app/models/provider/tiingo.rb:245

        faraday.request :json
        faraday.response :raise_error
        faraday.headers["Authorization"] = "Token #{api_key}"
        faraday.headers["Content-Type"] = "application/json"
      end
    end

    # Adds hourly request counter on top of the interval throttle from RateLimitable.
    def throttle_request
      super

      # Global per-hour request counter via cache (Redis).
      # Atomic increment-then-check avoids the TOCTOU of read-check-increment.
      hour_key = "tiingo:requests:#{Time.current.to_i / 3600}"
      new_count = Rails.cache.increment(hour_key, 1, expires_in: 7200.seconds).to_i

      if new_count >= max_requests_per_hour
        raise RateLimitError, "Tiingo hourly request limit reached (#{new_count}/#{max_requests_per_hour})"
      end
    end

    # Tracks unique symbols queried per month to stay within Tiingo's 500 symbols/month limit.
    # Uses atomic set-if-absent (Redis SETNX) to eliminate the read-then-write race
    # where two concurrent workers could both see the symbol as untracked and both
    # increment the counter.
    def track_symbol(symbol)
      symbol_key = "tiingo:symbol:#{Date.current.strftime('%Y-%m')}:#{symbol.upcase}"
      count_key  = "tiingo:symbol_count:#{Date.current.strftime('%Y-%m')}"

      # Atomic write-if-absent: returns false when the key already exists (Redis SETNX).
      # Only the first worker to claim this symbol will proceed to increment the counter.
      return unless Rails.cache.write(symbol_key, true, expires_in: 35.days, unless_exist: true)

      new_count = Rails.cache.increment(count_key, 1, expires_in: 35.days).to_i

      if new_count >= MAX_SYMBOLS_PER_MONTH

View on GitHub (pinned to e69894adb9)