Class: Findbug::Storage::RedisBuffer

Inherits:
Object
  • Object
show all
Defined in:
lib/findbug/storage/redis_buffer.rb

Overview

RedisBuffer provides fast, non-blocking writes to Redis.

THIS IS THE KEY TO ZERO PERFORMANCE IMPACT

Traditional error tracking (synchronous):

Request starts
    ↓
Exception occurs
    ↓
BLOCKING: Write to database (50-100ms)  ← Your user waits!
    ↓
Request ends

Findbug (asynchronous):

Request starts
    ↓
Exception occurs
    ↓
NON-BLOCKING: Spawn thread to write to Redis (0ms)
    ↓                          ↓
Request ends            Background: Redis write (1-2ms)
    ↓
User gets response immediately

WHY REDIS INSTEAD OF DATABASE?

Redis write: ~1-2ms Database write: ~50-100ms (with indexes, constraints, etc.)

Even if we made DB writes async, Redis is still better for buffering because:

  1. It’s faster (in-memory)

  2. It handles high write loads gracefully

  3. It has built-in expiration (TTL)

  4. It supports atomic list operations

The database is for long-term storage. Redis is for the fast buffer.

WHY Thread.new INSTEAD OF SIDEKIQ?

Sidekiq itself writes to Redis. If we used Sidekiq to buffer our errors:

  1. We’d add Sidekiq job overhead (~5ms)

  2. We’d share Redis connections with Sidekiq

  3. We’d depend on Sidekiq being healthy

A simple Thread.new is:

  1. Instant (no queue overhead)

  2. Independent of your job system

  3. Simpler (no job serialization)

We use Sidekiq/ActiveJob later for PERSISTING to DB, not for buffering.

Constant Summary collapse

ERRORS_KEY =

Key prefix for error events

"findbug:errors"
PERFORMANCE_KEY =

Key prefix for performance events

"findbug:performance"
STATS_KEY =

Key for tracking stats

"findbug:stats"

Class Method Summary collapse

Class Method Details

.clear! ⇒ Object

Clear all buffers (for testing)



149
150
151
152
153
154
155
 
# File 'lib/findbug/storage/redis_buffer.rb', line 149

def clear!
  ConnectionPool.with do |redis|
    redis.del(ERRORS_KEY, PERFORMANCE_KEY)
  end
rescue StandardError
  # Ignore errors during cleanup
end

.pop_errors(batch_size = 100) ⇒ Array<Hash>

Pop a batch of error events from the buffer

This is called by the PersistJob to move data from Redis to DB. It uses LPOP in a loop to get events atomically.

Parameters:

  • batch_size (Integer) (defaults to: 100) —

    maximum number of events to retrieve

Returns:

  • (Array<Hash>) —

    array of error events



111
112
113
 
# File 'lib/findbug/storage/redis_buffer.rb', line 111

def pop_errors(batch_size = 100)
  pop_batch(ERRORS_KEY, batch_size)
end

.pop_performance(batch_size = 100) ⇒ Array<Hash>

Pop a batch of performance events from the buffer

Parameters:

  • batch_size (Integer) (defaults to: 100) —

    maximum number of events to retrieve

Returns:

  • (Array<Hash>) —

    array of performance events



120
121
122
 
# File 'lib/findbug/storage/redis_buffer.rb', line 120

def pop_performance(batch_size = 100)
  pop_batch(PERFORMANCE_KEY, batch_size)
end

.push_error(event_data) ⇒ Object

Push an error event to the buffer (async, non-blocking)

IMPORTANT: This returns IMMEDIATELY. The actual write happens in a background thread. This is what makes us non-blocking.

Examples:

RedisBuffer.push_error({
  exception_class: "RuntimeError",
  message: "Something went wrong",
  backtrace: [...],
  context: {...}
})

Parameters:

  • event_data (Hash) —

    the error event data



91
92
93
 
# File 'lib/findbug/storage/redis_buffer.rb', line 91

def push_error(event_data)
  push_async(ERRORS_KEY, event_data)
end

.push_performance(event_data) ⇒ Object

Push a performance event to the buffer (async, non-blocking)

Parameters:

  • event_data (Hash) —

    the performance event data



99
100
101
 
# File 'lib/findbug/storage/redis_buffer.rb', line 99

def push_performance(event_data)
  push_async(PERFORMANCE_KEY, event_data)
end

.stats ⇒ Hash

Get buffer statistics (for monitoring)

Returns:

  • (Hash) —

    buffer stats including queue lengths



128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
 
# File 'lib/findbug/storage/redis_buffer.rb', line 128

def stats
  ConnectionPool.with do |redis|
    {
      error_queue_length: redis.llen(ERRORS_KEY),
      performance_queue_length: redis.llen(PERFORMANCE_KEY),
      circuit_breaker_state: CircuitBreaker.state,
      circuit_breaker_failures: CircuitBreaker.failure_count
    }
  end
rescue StandardError => e
  # Always return circuit breaker state even if Redis is down
  {
    error_queue_length: 0,
    performance_queue_length: 0,
    circuit_breaker_state: Findbug::Storage::CircuitBreaker.state,
    circuit_breaker_failures: Findbug::Storage::CircuitBreaker.failure_count,
    error: "Redis connection failed: #{e.message}"
  }
end