Class: Postburner::Configuration

Inherits:
Object
  • Object
show all
Defined in:
lib/postburner/configuration.rb

Overview

Configuration system for Postburner workers and connections.

Supports both programmatic configuration and YAML file loading. Configuration can be set per-environment and controls worker behavior, Beanstalkd connection, and execution strategies.

Examples:

Programmatic configuration

Postburner.configure do |config|
  config.beanstalk_url = 'beanstalk://localhost:11300'
  config.logger = Rails.logger
  config.worker_config = { name: 'default', queues: ['default'], forks: 2, threads: 10 }
end

Loading from YAML

config = Postburner::Configuration.load_yaml('config/postburner.yml', 'production', 'imports')

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Configuration

Returns a new instance of Configuration.

Parameters:

  • options (Hash) (defaults to: {}) —

    Configuration options

Options Hash (options):

  • :beanstalk_url (String) —

    Beanstalkd URL (default: ENV or localhost)

  • :logger (Logger) —

    Logger instance (default: Rails.logger)

  • :default_priority (Integer) —

    Default job priority (default: 65536, lower = higher priority)

  • :default_ttr (Integer) —

    Default time-to-run in seconds (default: 300)

  • :default_mailer_queue (String) —

    Queue name for Postburner::Mailer jobs (default: nil, uses default_queue)

  • :default_scheduler_interval (Integer) —

    Scheduler check interval in seconds (default: 300)

  • :default_scheduler_priority (Integer) —

    Scheduler job priority (default: 100)

  • :enqueue_options (Proc) —

    Proc that receives job and returns options hash with :priority and/or :ttr keys. Called during enqueue to customize job options. Priority cascade: job.priority > hook > class-level > default

  • :worker_config (Hash) —

    Worker configuration hash with keys:

    • :name [String] Worker name

    • :queues [Array<String>] Queue/tube names to process

    • :forks [Integer] Number of forked processes (0 = single process mode)

    • :threads [Integer] Number of threads per fork

    • :gc_limit [Integer, nil] Jobs to process before restart (nil = unlimited)

    • :timeout [Integer] Reserve command timeout in seconds (1-10, default: 3)

    • :shutdown_timeout [Integer] Seconds to wait for graceful shutdown (default: default_ttr)



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/postburner/configuration.rb', line 52

def initialize(options = {})
  @beanstalk_url = options[:beanstalk_url] || ENV['BEANSTALK_URL'] || 'beanstalk://localhost:11300'
  @logger = options[:logger] || (defined?(Rails) ? Rails.logger : Logger.new(STDOUT))
  @default_priority = options[:default_priority] || 65536
  @default_ttr = options[:default_ttr] || 300
  @default_queue = options[:default_queue] || 'default'
  @default_mailer_queue = options[:default_mailer_queue]
  @default_max_retries = options[:default_max_retries] || 0
  @default_retry_delay = options[:default_retry_delay] || ->(n) { 2 ** n }
  @default_scheduler_interval = options[:default_scheduler_interval] || 300
  @default_scheduler_priority = options[:default_scheduler_priority] || 100
  @enqueue_options = options[:enqueue_options]
  @worker_config = options[:worker_config] || {
    name: 'default',
    queues: ['default'],
    forks: 0,
    threads: 1,
    gc_limit: nil,
    timeout: 3,
    shutdown_timeout: @default_ttr
  }
end

Instance Attribute Details

#beanstalk_url ⇒ Object

Global settings



24
25
26
 
# File 'lib/postburner/configuration.rb', line 24

def beanstalk_url
  @beanstalk_url
end

#default_mailer_queue ⇒ Object

Returns the value of attribute default_mailer_queue.



25
26
27
 
# File 'lib/postburner/configuration.rb', line 25

def default_mailer_queue
  @default_mailer_queue
end

#default_max_retries ⇒ Object

Returns the value of attribute default_max_retries.



25
26
27
 
# File 'lib/postburner/configuration.rb', line 25

def default_max_retries
  @default_max_retries
end

#default_priority ⇒ Object

Global settings



24
25
26
 
# File 'lib/postburner/configuration.rb', line 24

def default_priority
  @default_priority
end

#default_queue ⇒ Object

Returns the value of attribute default_queue.



25
26
27
 
# File 'lib/postburner/configuration.rb', line 25

def default_queue
  @default_queue
end

#default_retry_delay ⇒ Object

Returns the value of attribute default_retry_delay.



25
26
27
 
# File 'lib/postburner/configuration.rb', line 25

def default_retry_delay
  @default_retry_delay
end

#default_scheduler_interval ⇒ Object

Returns the value of attribute default_scheduler_interval.



26
27
28
 
# File 'lib/postburner/configuration.rb', line 26

def default_scheduler_interval
  @default_scheduler_interval
end

#default_scheduler_priority ⇒ Object

Returns the value of attribute default_scheduler_priority.



26
27
28
 
# File 'lib/postburner/configuration.rb', line 26

def default_scheduler_priority
  @default_scheduler_priority
end

#default_ttr ⇒ Object

Global settings



24
25
26
 
# File 'lib/postburner/configuration.rb', line 24

def default_ttr
  @default_ttr
end

#enqueue_options ⇒ Object

Returns the value of attribute enqueue_options.



27
28
29
 
# File 'lib/postburner/configuration.rb', line 27

def enqueue_options
  @enqueue_options
end

#logger ⇒ Object

Global settings



24
25
26
 
# File 'lib/postburner/configuration.rb', line 24

def logger
  @logger
end

#worker_config ⇒ Object

Worker-specific settings (loaded for a single worker)



30
31
32
 
# File 'lib/postburner/configuration.rb', line 30

def worker_config
  @worker_config
end

Class Method Details

.load_yaml(path, env = 'development', worker_name = nil) ⇒ Configuration

Loads configuration from a YAML file with ERB support.

Supports ERB templating like Rails’ database.yml, allowing environment variables and Ruby expressions in configuration.

Examples:

Single worker (auto-selected)

config = Postburner::Configuration.load_yaml('config/postburner.yml', 'production')

Multiple workers (must specify)

config = Postburner::Configuration.load_yaml('config/postburner.yml', 'production', 'imports')

config/postburner.yml with ERB


default: &default
  beanstalk_url: <%= ENV.fetch('BEANSTALK_URL', 'beanstalk://localhost:11300') %>
  default_priority: 131072 # change default priority from 65536 to 131072
  default_mailer_queue: mailers  # optional: route Postburner::Mailer to separate queue

production: # <- environment config, i.e. defaults
  <<: *default
  default_forks: 2
  default_threads: 10
  default_gc_limit: 5000
  default_ttr: 300
  default_shutdown_timeout: 300  # Defaults to default_ttr if not set
  workers: # <- worker configs
    imports: # <- worker name
      timeout: 3           # Reserve timeout in seconds (1-10, default: 3)
                           # Lower values enable faster graceful shutdowns
      forks: 4             # Overrides default_forks
      threads: 1           # Overrides default_threads
      gc_limit: 500        # Overrides default_gc_limit
      shutdown_timeout: 600  # Wait 10 min for long imports to finish
      queues:
        - imports
        - data_processing

Parameters:

  • path (String) —

    Path to YAML configuration file

  • env (String) (defaults to: 'development') —

    Environment name (e.g., ‘production’, ‘development’)

  • worker_name (String, nil) (defaults to: nil) —

    Worker name to load (nil = auto-select if single worker)

Returns:

Raises:

  • (Errno::ENOENT) —

    if file doesn’t exist

  • (ArgumentError) —

    if environment not found in YAML

  • (ArgumentError) —

    if multiple workers defined but worker_name not specified

  • (ArgumentError) —

    if worker_name specified but not found



123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
 
# File 'lib/postburner/configuration.rb', line 123

def self.load_yaml(path, env = 'development', worker_name = nil)
  yaml = YAML.load(ERB.new(File.read(path)).result, aliases: true)
  # env_config = top-level environment config (development:, production:, etc.)
  env_config = yaml[env.to_s] || yaml[env.to_sym]

  raise ArgumentError, "Environment '#{env}' not found in #{path}" unless env_config

  workers = env_config['workers']
  raise ArgumentError, "No 'workers:' section found in #{path} for environment '#{env}'" unless workers

  # Auto-select single worker or validate worker_name
  if worker_name.nil?
    if workers.size == 1
      worker_name = workers.keys.first
    else
      raise ArgumentError, <<~ERROR
        Configuration has multiple workers, but --worker not specified
        Available workers: #{workers.keys.join(', ')}
        Usage: bin/postburner --worker <name>
      ERROR
    end
  else
    unless workers.key?(worker_name)
      raise ArgumentError, <<~ERROR
        Worker '#{worker_name}' not found in #{path}
        Available workers: #{workers.keys.join(', ')}
      ERROR
    end
  end

  # worker_yaml = specific worker configuration from YAML (workers: imports:)
  worker_yaml = workers[worker_name]

  # Build worker_config hash - worker-level overrides env-level defaults
  default_ttr = (env_config['default_ttr'] || 300).to_i
  worker_config = {
    name: worker_name,
    queues: worker_yaml['queues'] || ['default'],
    forks: (worker_yaml['forks'] || env_config['forks'] || 0).to_i,
    threads: (worker_yaml['threads'] || env_config['threads'] || 1).to_i,
    gc_limit: (worker_yaml['gc_limit'] || env_config['gc_limit'])&.to_i,
    timeout: (worker_yaml['timeout'] || 3).to_i,
    shutdown_timeout: (worker_yaml['shutdown_timeout'] || env_config['shutdown_timeout'] || default_ttr).to_i
  }

  options = {
    beanstalk_url: env_config['beanstalk_url'],
    default_priority: env_config['default_priority'],
    default_ttr: env_config['default_ttr'],
    default_queue: env_config['default_queue'],
    default_mailer_queue: env_config['default_mailer_queue'],
    default_scheduler_interval: env_config['scheduler_interval'],
    default_scheduler_priority: env_config['scheduler_priority'],
    worker_config: worker_config
  }

  new(options)
end

Instance Method Details

#expand_tube_name(queue_name, env = nil) ⇒ String

Expands queue name to full tube name with environment prefix.

Converts a simple queue name (e.g., ‘default’, ‘critical’) to the full Beanstalkd tube name with environment namespace (e.g., ‘postburner.production.critical’).

Examples:

config.expand_tube_name('critical', 'production')
# => "postburner.production.critical"

config.expand_tube_name('default')
# => "postburner.test.default" (in test env)

Parameters:

  • queue_name (String, Symbol) —

    Base queue name

  • env (String, Symbol, nil) (defaults to: nil) —

    Environment name (defaults to Rails.env)

Returns:

  • (String) —

    Full tube name with environment prefix



211
212
213
214
215
216
217
 
# File 'lib/postburner/configuration.rb', line 211

def expand_tube_name(queue_name, env = nil)
  env ||= defined?(Rails) ? Rails.env : nil
  [
    tube_prefix(env),
    queue_name,
  ].compact.join('.')
end

#expanded_tube_names(env = nil) ⇒ Array<String>

Returns array of expanded tube names with environment prefix.

Examples:

config.expanded_tube_names('production')  # => ['postburner.production.imports', 'postburner.production.data_processing']

Parameters:

  • env (String, Symbol, nil) (defaults to: nil) —

    Environment name (defaults to Rails.env or ‘development’)

Returns:

  • (Array<String>) —

    Array of expanded tube names



253
254
255
 
# File 'lib/postburner/configuration.rb', line 253

def expanded_tube_names(env = nil)
  queue_names.map { |q| expand_tube_name(q, env) }
end

#queue_names ⇒ Array<String>

Returns array of queue names from worker config.

Examples:

config.queue_names  # => ['imports', 'data_processing']

Returns:

  • (Array<String>) —

    Queue names



189
190
191
 
# File 'lib/postburner/configuration.rb', line 189

def queue_names
  @worker_config[:queues].map(&:to_s)
end

#scheduler_tube_name(env = nil) ⇒ String

Returns the scheduler tube name with environment prefix.

Examples:

config.scheduler_tube_name
# => 'postburner.production.scheduler'

Parameters:

  • env (String, nil) (defaults to: nil) —

    Environment name (defaults to Rails.env)

Returns:

  • (String) —

    Expanded scheduler tube name



267
268
269
 
# File 'lib/postburner/configuration.rb', line 267

def scheduler_tube_name(env = nil)
  expand_tube_name(Postburner::Scheduler::SCHEDULER_TUBE_NAME, env)
end

#tube_prefix(env = nil) ⇒ String

Note:

Do not set ActiveJob’s ‘queue_name_prefix` - Postburner handles queue name expansion automatically via the adapter.

Returns the Beanstalkd tube name prefix for the given environment.

Postburner automatically prefixes all queue names with this value. The ActiveJob adapter expands queue names using this prefix, so ‘queue_as :default` becomes `postburner.production.default` in production.

Examples:

config.tube_prefix('production')  # => "postburner.production"
config.tube_prefix                # => "postburner.test" (in test env)

Parameters:

  • env (String, Symbol, nil) (defaults to: nil) —

    Environment name (defaults to Rails.env or nil)

Returns:

  • (String) —

    Tube prefix (e.g., “postburner.production”)



236
237
238
239
240
241
242
 
# File 'lib/postburner/configuration.rb', line 236

def tube_prefix(env = nil)
  env ||= defined?(Rails) ? Rails.env : nil
  [
    'postburner',
    env,
  ].compact.join('.')
end