Class: Phronomy::Concurrency::OffloadPool

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/engine/concurrency/offload_pool.rb

Overview

A bounded, observable thread pool for synchronous work that must not run on the Runtime EventLoop.

Architectural boundary

OffloadPool is the bounded OS-thread execution boundary for synchronous operations that would otherwise occupy the EventLoop for too long. The work may be blocking I/O, CPU-bound Ruby processing, or another application-defined synchronous call. Phronomy deliberately does not classify the workload by cause; the application decides whether a unit of work is EventLoop-safe.

Logical waits are different. Waiting for another Phronomy Task, Agent, Workflow, ToolInvocation, timer, or FSMSession must remain an explicit EventLoop/FSMSession continuation and must not consume an OffloadPool worker. See ADR-010.

Submitted work is bounded so that:

  1. The total number of worker OS threads is capped.
  2. Queue depth is bounded (backpressure when the pool is saturated).
  3. Per-operation timeouts and cancellation settle the caller-facing Task.
  4. Operations that settle after worker execution has started are tracked as abandoned until that worker returns.
  5. Metrics expose active work, queue depth, cumulative abandonment, currently-active abandonment, and average queue wait time.

OffloadPool does not provide CPU isolation, CPU parallelism guarantees, or fairness between I/O and CPU-heavy work classes. Applications that need resource isolation may use named pools via Runtime#pool.

Examples:

Submitting synchronous work

task = runtime.offload.submit(timeout: 30) { expensive_call }
result = task.wait_result

With cancellation

token = Phronomy::Concurrency::CancellationToken.timeout_after(60)
task = pool.submit(timeout: 30, cancellation_token: token) { expensive_call }
result = task.wait_result

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(pool_size: 10, queue_size: 100, name: nil, logger: nil, timer_queue_provider: nil) ⇒ OffloadPool

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of OffloadPool.

Parameters:

  • pool_size (Integer) (defaults to: 10) —

    maximum number of worker threads

  • queue_size (Integer) (defaults to: 100) —

    maximum pending operations waiting for a worker

  • name (String, Symbol, nil) (defaults to: nil) —

    optional pool name used in thread labels

  • logger (Logger, nil) (defaults to: nil) —

    optional logger for warnings

  • timer_queue_provider (#call, nil) (defaults to: nil) —

    returns a TimerQueue-compatible object. Required when submit(timeout:) or a monotonic-deadline cancellation token is used.



290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 290

def initialize(
  pool_size: 10,
  queue_size: 100,
  name: nil,
  logger: nil,
  timer_queue_provider: nil
)
  @pool_size = pool_size
  @queue_size = queue_size
  @name = name
  @logger = logger
  @timer_queue_provider = timer_queue_provider
  @queue = SizedQueue.new(queue_size)
  @active_count = 0
  @abandoned_count = 0
  @running_operation_ids = {}
  @abandoned_active_operation_ids = {}
  @total_wait_ns = 0
  @completed_count = 0
  @mutex = Mutex.new
  @shutdown = false
  @workers = Array.new(pool_size) { |i| spawn_worker(i) }
end

Instance Attribute Details

#name ⇒ String, ... (readonly)

Returns pool name used in thread labels.

Returns:

  • (String, Symbol, nil) —

    pool name used in thread labels



524
525
526
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 524

def name
  @name
end

#pool_size ⇒ Integer (readonly)

Returns configured maximum number of worker threads.

Returns:

  • (Integer) —

    configured maximum number of worker threads



518
519
520
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 518

def pool_size
  @pool_size
end

#queue_size ⇒ Integer (readonly)

Returns configured maximum queue depth.

Returns:

  • (Integer) —

    configured maximum queue depth



521
522
523
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 521

def queue_size
  @queue_size
end

Instance Method Details

#abandoned_active_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns number of abandoned operations that still occupy worker capacity at this instant.

Returns:

  • (Integer) —

    number of abandoned operations that still occupy worker capacity at this instant



501
502
503
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 501

def abandoned_active_count
  @mutex.synchronize { @abandoned_active_operation_ids.size }
end

#abandoned_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns cumulative number of operations whose caller-facing timeout or cancellation settled after worker execution had started.

Returns:

  • (Integer) —

    cumulative number of operations whose caller-facing timeout or cancellation settled after worker execution had started



494
495
496
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 494

def abandoned_count
  @mutex.synchronize { @abandoned_count }
end

#active_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns number of operations currently executing on workers.

Returns:

  • (Integer) —

    number of operations currently executing on workers



481
482
483
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 481

def active_count
  @mutex.synchronize { @active_count }
end

#average_wait_seconds ⇒ Float

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Average time (in seconds) that completed or skipped operations spent in the queue waiting for a worker. Returns 0.0 when none have been processed yet.

Returns:

  • (Float) 


509
510
511
512
513
514
515
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 509

def average_wait_seconds
  @mutex.synchronize do
    return 0.0 if @completed_count.zero?

    @total_wait_ns / @completed_count.to_f / 1_000_000_000.0
  end
end

#queue_depth ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns number of operations waiting in the queue.

Returns:

  • (Integer) —

    number of operations waiting in the queue



487
488
489
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 487

def queue_depth
  @queue.size
end

#shutdown(drain_timeout: 30) ⇒ self

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Gracefully drains the pool and terminates all worker threads. Waits up to drain_timeout seconds for in-flight operations to finish.

Closing the underlying SizedQueue signals workers to exit after draining remaining items, without blocking on a full-queue push.

Parameters:

  • drain_timeout (Numeric) (defaults to: 30) —

    seconds to wait for workers to finish

Returns:

  • (self) 


470
471
472
473
474
475
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 470

def shutdown(drain_timeout: 30)
  @shutdown = true
  @queue.close
  @workers.each { |thread| thread.join(drain_timeout) }
  self
end

#submit(timeout: nil, cancellation_token: nil, on_full: :wait, full_timeout: nil) { ... } ⇒ Phronomy::Task

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits synchronous off-EventLoop work to the pool.

Returns a Task immediately after queue admission; the block runs on a worker thread. Do not submit logical waits (for example waiting for a child Agent Task) merely to make them asynchronous; those belong to FSMSession/EventLoop completion events.

A submit-time timeout is an operation-wide deadline measured from the start of this method, including queue wait. The timer settles the Task and notifies on_complete without forcibly interrupting a running worker. If the deadline fires before worker execution starts, the block is skipped. If it fires after execution starts, the private Operation is marked abandoned and the eventual worker result is discarded.

The submit cancellation_token is also operation-wide. Explicit cancellation settles the Task immediately. A token with a monotonic deadline is attached to the Runtime timer queue so deadline expiry becomes explicit cancellation without adding a polling Thread. Cancellation before execution skips the block; cancellation after execution starts abandons only the caller-facing result and never uses Thread#raise.

Synchronous queue admission may delay return from this method when on_full: :wait is used. EventLoop-owned framework paths therefore submit with on_full: :raise and handle backpressure asynchronously.

Parameters:

  • timeout (Numeric, nil) (defaults to: nil) —

    operation-wide deadline in seconds

  • cancellation_token (CancellationToken, nil) (defaults to: nil) —

    operation-wide token

  • on_full (Symbol) (defaults to: :wait) —

    :wait, :raise, or :timeout

  • full_timeout (Numeric, nil) (defaults to: nil) —

    queue-admission timeout for on_full: :timeout

Yields:

  • block containing synchronous work

Returns:

Raises:



351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 351

def submit(
  timeout: nil,
  cancellation_token: nil,
  on_full: :wait,
  full_timeout: nil,
  &block
)
  raise Phronomy::PoolShutdownError, "pool has been shut down" if @shutdown

  submitted_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  already_cancelled = cancellation_token&.cancelled? || false
  cancellation_remaining = if already_cancelled
    nil
  else
    cancellation_token&.remaining_monotonic_seconds
  end

  needs_timer = !timeout.nil? ||
    (!cancellation_remaining.nil? && cancellation_remaining > 0)
  timer_queue = @timer_queue_provider&.call if needs_timer
  if needs_timer && !timer_queue
    raise Phronomy::ConfigurationError,
      "timer_queue is required when submit timeout or cancellation deadline is specified"
  end

  operation = Operation.new(
    block,
    timeout: timeout,
    cancellation_token: cancellation_token,
    submitted_at: submitted_at,
    task_name: offload_task_name,
    on_abandoned: method(:record_abandoned)
  )
  task = operation.task

  # on_cancel only reacts to explicit cancel!, whereas cancelled? also covers a
  # monotonic deadline. Promote an already-expired deadline immediately.
  if already_cancelled
    cancellation_token.cancel!
    return task
  end

  begin
    if timeout
      elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - submitted_at
      remaining = timeout.to_f - elapsed
      if remaining <= 0
        operation.fire_timeout!
        return task
      end

      # Arm before queue admission so the deadline includes time spent waiting
      # for a queue slot.
      timer_queue.schedule(seconds: remaining) { operation.fire_timeout! }
    end

    if cancellation_remaining
      # Re-read after timeout setup so the scheduled delay reflects setup time.
      remaining = cancellation_token.remaining_monotonic_seconds
      if remaining <= 0
        cancellation_token.cancel!
        return task
      end
      timer_queue.schedule(seconds: remaining) { cancellation_token.cancel! }
    end

    # Cancellation/timeout can race with timer registration. Do not enqueue
    # already-settled work when the race is observable here.
    return task if operation.settled?

    case on_full
    when :raise
      begin
        @queue.push(operation, true)
      rescue ThreadError
        raise Phronomy::BackpressureError,
          "OffloadPool queue is full (depth: #{@queue_size})"
      end
    when :timeout
      deadline = full_timeout ?
        (Process.clock_gettime(Process::CLOCK_MONOTONIC) + full_timeout) :
        nil
      loop do
        return task if operation.settled?

        @queue.push(operation, true)
        break
      rescue ThreadError
        if deadline &&
            Process.clock_gettime(Process::CLOCK_MONOTONIC) >= deadline
          raise Phronomy::TimeoutError,
            "timed out waiting for a free slot in OffloadPool"
        end
        sleep(0.005)
      end
    else # :wait (default)
      @queue.push(operation)
    end
  rescue ClosedQueueError => e
    # Shutdown raced with this submit — preserve the existing public error.
    operation.fail_submission!(e)
    raise Phronomy::PoolShutdownError, "pool has been shut down"
  rescue => e
    operation.fail_submission!(e)
    raise
  end

  task
end