Class: Sidekiq::Fiber::Processor

Inherits:
Processor
  • Object
show all
Defined in:
lib/sidekiq/fiber/processor.rb

Overview

A Sidekiq::Processor replacement that runs fiber-aware jobs as fibers inside a per-thread Async event loop.

One event loop runs per thread. Each fiber job is scheduled as a child task inside that loop. The thread fetches jobs continuously, scheduling each as a fiber without waiting for previous fibers to finish.

A semaphore bounds concurrent fibers per thread. This directly controls how many DB connections this processor can consume:

max_connections = threads * fiber_concurrency

On normal shutdown (@done = true), the fetch loop exits but the event loop waits for all in-flight fibers to complete before returning.

On hard shutdown (Sidekiq::Shutdown raised), in-flight fibers that have not completed are requeued — identical to Sidekiq’s default behaviour.

Instance Method Summary collapse

Constructor Details

#initialize(capsule, &block) ⇒ Processor

Returns a new instance of Processor.



24
25
26
27
28
29
30
31
 
# File 'lib/sidekiq/fiber/processor.rb', line 24

def initialize(capsule, &block)
  super
  @fiber_concurrency  = capsule.config[:fiber_concurrency] || 100
  @active_fibers      = 0
  @active_fibers_lock = Mutex.new
  stats_pool = capsule.config.new_redis_pool(@fiber_concurrency, "sidekiq-fiber-stats")
  @stats = Stats.new(stats_pool)
end