Class: RailsErrorDashboard::Services::RackAttackTracker

Inherits:
Object
  • Object
show all
Defined in:
lib/rails_error_dashboard/services/rack_attack_tracker.rb

Overview

Buffers Rack::Attack events in a thread-local hash and flushes them to the database asynchronously.

WHY THIS EXISTS (issue #143): Rack::Attack events were previously only recorded as breadcrumbs. Breadcrumbs are harvested exclusively by LogError, so an event was only ever persisted if an unrelated exception happened to be raised later in the same request. A throttled request returns HTTP 429 and raises nothing, so the event was always discarded when ErrorCatcher cleared the buffer. This tracker persists events independently of error capture.

Events are aggregated by (rule, match_type, discriminator, path, method) and counted, rather than stored one row per event — a rate-limit flood is exactly when we must not do one INSERT per request.

SAFETY RULES (HOST_APP_SAFETY.md):

  • Zero I/O in the record path (hash lookup + integer increment)
  • Never raises — every public method wrapped in rescue
  • Thread-local state, no mutex needed
  • LRU eviction bounds memory (rotating-IP attacks cannot grow it unbounded)
  • Async flush via background job

Constant Summary collapse

COUNTS_THREAD_KEY = 
:red_rack_attack_counts
DEADLINE_THREAD_KEY =

Monotonic timestamp of the moment the buffer became non-empty — the DEADLINE clock, not a "last flush" clock.

WHY THE DISTINCTION MATTERS: this used to hold the last flush time and be seeded lazily inside maybe_flush! with ||= now, which meant the very first event of a buffer set the clock to now and then compared `now - now

interval` — false, always. A rule that matched once and never again

therefore never flushed at all, and a manual curl test showed an empty table indefinitely (issue #170, third report). Seeding when the buffer STARTS filling makes the guarantee "buffered data is never older than flush_interval", which is the property the dashboard actually needs.

:red_rack_attack_deadline_at
FLUSH_THREAD_KEY =

Kept as an alias so a host or spec holding the old key name still clears the right slot. Both are cleared together in reset!.

:red_rack_attack_last_flush
MAX_RULE_LENGTH =

Field length caps — must match the column limits in the migration so that truncation happens before the value ever reaches the unique upsert index.

250
MAX_DISCRIMINATOR_LENGTH = 
191
MAX_PATH_LENGTH = 
191
MAX_METHOD_LENGTH = 
10
MAX_USER_AGENT_LENGTH = 
191
DEFAULT_FLUSH_INTERVAL =

Reserved rule/match_type used to account for counts dropped by LRU eviction. Without this the evicted count vanishes silently and the dashboard under-reports with no indication anything was lost — the same problem StormProtection::CountBuffer solves with an overflow counter. Fallback when configuration is unreadable. Must match Configuration#rack_attack_flush_interval's default.

5
OVERFLOW_RULE = 
"__overflow__"
OVERFLOW_MATCH_TYPE = 
"overflow"
KEY_SEPARATOR =

Separator for the composite buffer key. Chosen because it cannot appear in an HTTP method and is vanishingly unlikely in a rule name or path.

""

Class Method Summary collapse

Class Method Details

.buffered_counts ⇒ Object

Current buffered counts (inspection / specs). Non-destructive.



236
237
238
239
240
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 236

def buffered_counts
  (Thread.current[COUNTS_THREAD_KEY] || {}).dup
rescue => e
  {}
end

.flush!(sync: false) ⇒ Object

Flush buffered counts to the database (async by default). Clears the thread-local buffer before dispatching so a slow/failed flush cannot double-count on the next call.



122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 122

def flush!(sync: false)
  counts = Thread.current[COUNTS_THREAD_KEY]
  return if counts.nil? || counts.empty?

  snapshot = counts.dup
  counts.clear
  # Buffer is empty again, so there is nothing to be late: clear the
  # deadline. The next record! reseeds it. Leaving a stale timestamp
  # here would make the very next event look instantly overdue.
  Thread.current[DEADLINE_THREAD_KEY] = nil
  Thread.current[FLUSH_THREAD_KEY] = nil

  dispatch_flush(snapshot, sync: sync)
  nil
rescue => e
  RailsErrorDashboard::Logger.debug(
    "[RailsErrorDashboard] RackAttackTracker.flush! failed: #{e.class} - #{e.message}"
  )
  nil
end

.flush_all_threads! ⇒ Object

Flush every live thread's buffer, not just the caller's.

WHY: flush! only ever sees Thread.current. Buffers live on the Puma threads that served the requests, so at shutdown (and from a background job) the caller's own buffer is empty while the real counts sit on threads nobody is asking. Without this, everything buffered at SIGTERM is lost, and a rule that matches once and then sees no further traffic on that thread is never persisted at all.

Thread#[] reads another thread's fiber-locals directly, so no thread registry is needed — the same approach SwallowedExceptionTracker uses. sync: true because callers are already off the request path.



155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 155

def flush_all_threads!
  Thread.list.each do |thread|
    # Rescue per thread, not just around the whole loop: one thread
    # whose write fails must not strand the buffers of every thread
    # after it in the list.
    begin
      counts = thread[COUNTS_THREAD_KEY]
      next if counts.nil? || counts.empty?

      snapshot = counts.dup
      counts.clear
      thread[DEADLINE_THREAD_KEY] = nil
      thread[FLUSH_THREAD_KEY] = nil

      dispatch_flush(snapshot, sync: true)
    rescue => e
      RailsErrorDashboard::Logger.debug(
        "[RailsErrorDashboard] RackAttackTracker.flush_all_threads! skipped a thread: #{e.class} - #{e.message}"
      )
    end
  end
  nil
rescue => e
  RailsErrorDashboard::Logger.debug(
    "[RailsErrorDashboard] RackAttackTracker.flush_all_threads! failed: #{e.class} - #{e.message}"
  )
  nil
end

.flush_due? ⇒ Boolean

Cheap deadline check — a float subtraction, no I/O.

Returns true when the buffer has been waiting at least flush_interval. Uses a monotonic clock: Time.now can jump backwards (NTP correction, leap second) and would then defer the flush indefinitely.

Returns:

  • (Boolean) 


260
261
262
263
264
265
266
267
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 260

def flush_due?
  deadline = Thread.current[DEADLINE_THREAD_KEY]
  return false if deadline.nil?

  (monotonic_now - deadline) >= flush_interval
rescue => e
  false
end

.flush_if_due! ⇒ Object

Drain this thread's buffer at the end of a unit of work (a request or a job), if it has been waiting longer than flush_interval.

WHY THIS EXISTS (issue #170, third report): before this, the ONLY in-process drain was maybe_flush! inside record, so the buffer could only ever be flushed by a LATER event landing on the SAME thread. Two consequences, both reported as "no events are recorded at all":

1. A rule that matched once showed nothing until the process exited.
 `curl` once, look at the dashboard, see an empty table — forever.
2. Worse, Puma reuses and retires threads. Counts buffered on a thread
 that dies are unreachable to flush_all_threads! (it walks
 Thread.list), so they were lost outright, not merely delayed.
 Measured: 5 of 5 events lost when the serving threads exited.

ActiveSupport::Executor#to_complete is the right boundary because Rails already guarantees it runs once per request and once per job. Crucially, ActionDispatch::Executor returns a Rack::BodyProxy and defers the hook until the SERVER CLOSES THE RESPONSE BODY — so this runs after the client has its bytes and cannot delay the response (safety rule 2).

The flush is gated on flush_due?, so a flood does not turn into one UPDATE per request — the exact regression #143's buffer exists to prevent. Measured 0.068 ms/req gated vs 0.508 ms/req ungated.



208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 208

def flush_if_due!
  return unless enabled?
  return unless flush_due?

  # sync: the response is already sent, so there is nothing left to block,
  # and enqueueing a job per interval would be more overhead than the
  # single upsert it replaces.
  flush!(sync: true)
  nil
rescue => e
  RailsErrorDashboard::Logger.debug(
    "[RailsErrorDashboard] RackAttackTracker.flush_if_due! failed: #{e.class} - #{e.message}"
  )
  nil
end

.monotonic_now ⇒ Object 



269
270
271
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 269

def monotonic_now
  Process.clock_gettime(Process::CLOCK_MONOTONIC)
end

.parse_key(key) ⇒ Array<String>

Decompose a buffer key back into its parts.

The limit must match the field count exactly. With a limit of 5 the user agent would be glued onto http_method instead of standing alone. split also drops trailing empty fields without the limit, so a key whose user agent is blank must still yield six elements.

Returns:

  • (Array<String>) —

    [rule, match_type, discriminator, path, http_method, user_agent]



250
251
252
253
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 250

def parse_key(key)
  parts = key.to_s.split(KEY_SEPARATOR, 6)
  parts.fill("", parts.length, 6 - parts.length)
end

.record(rule:, match_type:, discriminator: nil, path: nil, http_method: nil, user_agent: nil) ⇒ Object

Record a single Rack::Attack event. Called from the AS::Notifications subscriber on every throttle/blocklist/track match.

Parameters:

  • rule (String) —

    matched rule name (env)

  • match_type (String) —

    "throttle" | "blocklist" | "track"

  • discriminator (String) (defaults to: nil) —

    rate-limit key (usually IP or user id)

  • path (String) (defaults to: nil) —

    request path

  • http_method (String) (defaults to: nil) —

    request method

  • user_agent (String) (defaults to: nil) —

    client user agent, for AI/crawler attribution



78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 78

def record(rule:, match_type:, discriminator: nil, path: nil, http_method: nil,
           user_agent: nil)
  return unless enabled?

  key = build_key(
    truncate(rule, MAX_RULE_LENGTH),
    match_type.to_s,
    truncate(discriminator, MAX_DISCRIMINATOR_LENGTH),
    truncate(path, MAX_PATH_LENGTH),
    truncate(http_method, MAX_METHOD_LENGTH),
    truncate(user_agent, MAX_USER_AGENT_LENGTH)
  )

  counts = (Thread.current[COUNTS_THREAD_KEY] ||= {})

  # Start the deadline the moment the buffer goes from empty to non-empty.
  # Doing it here (rather than lazily at flush-check time) is what makes
  # "never older than flush_interval" true for a buffer that receives
  # exactly one event and then goes quiet.
  Thread.current[DEADLINE_THREAD_KEY] ||= monotonic_now if counts.empty?

  counts[key] = (counts[key] || 0) + 1

  # LRU eviction — bounds memory under rotating-discriminator attacks.
  # Loops because the overflow bucket occupies a slot of its own once
  # created, so a single eviction may not bring the map back under cap.
  # evict_oldest! returns false once only the overflow key is left, which
  # guarantees termination even if max_cache_size is misconfigured to 0.
  while counts.size > max_cache_size
    break unless evict_oldest!(counts)
  end

  maybe_flush!
  nil
rescue => e
  RailsErrorDashboard::Logger.debug(
    "[RailsErrorDashboard] RackAttackTracker.record failed: #{e.class} - #{e.message}"
  )
  nil
end

.reset! ⇒ Object

Clear thread-local state without persisting. Used by specs and by thread teardown paths.



226
227
228
229
230
231
232
233
 
# File 'lib/rails_error_dashboard/services/rack_attack_tracker.rb', line 226

def reset!
  Thread.current[COUNTS_THREAD_KEY] = nil
  Thread.current[DEADLINE_THREAD_KEY] = nil
  Thread.current[FLUSH_THREAD_KEY] = nil
  nil
rescue => e
  nil
end