Class: Pikuri::Tool::Search::RateLimiter

Inherits:
Object
  • Object
show all
Defined in:
lib/pikuri/tool/search/rate_limiter.rb

Overview

Thread-safe pacing + circuit-breaker wrapper for a search provider.

#call { … } enforces a minimum interval between consecutive invocations of the block (sleeping if the previous one was too recent), and watches for Engines::Unavailable raised by the block: when that happens, a cooldown deadline is recorded and further calls within the window raise Engines::Unavailable immediately without invoking the block. This stops a provider that has been rate-limited or bot-blocked from being hammered with retries.

The mutex is held across the block, so concurrent callers serialize — matching the behavior DuckDuckGo has always required to keep its IP-spacing throttle correct under concurrent agents.

Uses wall-clock Time.now rather than the monotonic clock; the intervals here are 1s–5min, well above any realistic NTP step, and Time.now keeps tests trivially fakeable with Timecop.

Instance Method Summary collapse

Constructor Details

#initialize(min_interval:, cooldown:) ⇒ RateLimiter

Returns a new instance of RateLimiter.

Parameters:

  • min_interval (Float) —

    minimum seconds between consecutive block invocations. #call sleeps if a previous call was more recent.

  • cooldown (Float) —

    seconds to refuse calls after the block raises Engines::Unavailable. Calls within this window raise Engines::Unavailable immediately without invoking the block.



34
35
36
37
38
39
40
 
# File 'lib/pikuri/tool/search/rate_limiter.rb', line 34

def initialize(min_interval:, cooldown:)
  @min_interval = min_interval
  @cooldown = cooldown
  @mutex = Mutex.new
  @last_call_at = nil
  @cooldown_until = nil
end

Instance Method Details

#call ⇒ Object

Run the given block subject to throttle and cooldown rules.

The block is invoked with the mutex held, so concurrent calls serialize: only one block runs at a time per limiter instance. If the block raises Engines::Unavailable, the cooldown is armed and the exception is re-raised. Any other exception bubbles up without arming cooldown — only “try again later” signals from the provider are treated as backoff triggers.

Yield Returns:

  • (Object) —

    block’s return value is passed through

Returns:

  • (Object) —

    whatever the block returned

Raises:

  • (Engines::Unavailable) —

    either re-raised from the block, or raised directly when the limiter is currently in cooldown



55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'lib/pikuri/tool/search/rate_limiter.rb', line 55

def call
  @mutex.synchronize do
    now = Time.now
    if @cooldown_until && now < @cooldown_until
      remaining = (@cooldown_until - now).ceil
      raise Engines::Unavailable, "rate-limiter cooldown active for another #{remaining}s"
    end

    if @last_call_at
      elapsed = now - @last_call_at
      sleep_for(@min_interval - elapsed) if elapsed < @min_interval
    end
    @last_call_at = Time.now

    begin
      yield
    rescue Engines::Unavailable
      @cooldown_until = Time.now + @cooldown
      raise
    end
  end
end