resque-job-chain

A Resque plugin for declaring ordered chains of jobs with shared context, configurable error strategies, and atomic Redis state tracking.

Installation

gem 'resque-job-chain'

Usage

Declaring a chain

chain = Resque::JobChain.build("order-processing-#{order_id}") do |c|
  c.step FetchOrderJob, order_id: order_id
  c.step ValidateInventoryJob, order_id: order_id, on_failure: :skip
  c.step ChargePaymentJob, order_id: order_id, on_failure: :retry, max_attempts: 3
  c.step ShipOrderJob, order_id: order_id
  c.step SendConfirmationJob, order_id: order_id

  c.on_failure :abort
  c.context requested_at: Time.now.iso8601
end

chain.start!

Job classes

Extend Resque::Plugins::JobChain in your job classes to enable chain progression:

class ChargePaymentJob
  extend Resque::Plugins::JobChain
  @queue = :payments

  def self.perform(params)
    result = PaymentGateway.charge(params['order_id'], params['amount'])

    # Write to chain context for downstream steps
    Resque::JobChain.update_context(params['_chain_id'], payment_ref: result.reference)
  end
end

Each step receives a hash with:

• Its declared args
• The current chain context (accumulated from prior steps)
• _chain_id and _step_index metadata

Error strategies

Per-step or chain-level:

Strategy	Behavior
:abort	Mark chain as failed, stop execution (default)
:retry	Re-enqueue the step, up to max_attempts
:skip	Mark step as skipped, advance to next

Configuration

Resque::JobChain.configure do |c|
  c.default_strategy = :abort
  c.default_max_attempts = 3
  c.completed_chain_ttl = 86_400  # 24h
  c.statsd_client = MyStatsD
  c.logger = Rails.logger

  c.on(:chain_started) { |id| log("Chain #{id} started") }
  c.on(:step_completed) { |id, idx| log("Chain #{id} step #{idx} done") }
  c.on(:chain_completed) { |id| log("Chain #{id} completed") }
  c.on(:chain_failed) { |id, idx| alert("Chain #{id} failed at step #{idx}") }
end

Querying chain status

Resque::JobChain.status('order-processing-42')
# => {"status"=>"running", "current_step"=>"2", "total_steps"=>"5", ...}

Resque::JobChain.context('order-processing-42')
# => {"requested_at"=>"2024-01-01T00:00:00Z", "payment_ref"=>"pay_abc"}

Resque::JobChain.active_chains
# => ["order-processing-42", "order-processing-43"]

Design

Built using Gang of Four patterns:

• Builder for chain declaration DSL
• Command for step encapsulation
• Chain of Responsibility for sequential execution
• Strategy for pluggable error handling
• Template Method for execution lifecycle
• Observer for lifecycle callbacks

All state transitions are atomic via Redis Lua scripts.

Development

docker-compose up -d
REDIS_URL=redis://localhost:6380/15 bundle exec rspec
bundle exec rubocop

License

MIT