Class: Phronomy::Concurrency::OffloadPool

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/engine/concurrency/offload_pool.rb

Overview

A bounded, observable thread pool for synchronous work that must not run on the Runtime EventLoop.

Architectural boundary

OffloadPool is the bounded OS-thread execution boundary for synchronous operations that would otherwise occupy the EventLoop for too long. The work may be blocking I/O, CPU-bound Ruby processing, or another application-defined synchronous call. Phronomy deliberately does not classify the workload by cause; the application decides whether a unit of work is EventLoop-safe.

Logical waits are different. Waiting for another Phronomy Task, Agent, Workflow, ToolInvocation, timer, or FSMSession must remain an explicit EventLoop/FSMSession continuation and must not consume an OffloadPool worker. See ADR-010.

Submitted work is bounded so that:

  1. The total number of worker OS threads is capped.
  2. Queue depth is bounded (backpressure when the pool is saturated).
  3. Per-operation timeouts and cancellation settle the caller-facing handle.
  4. Operations that settle after worker execution has started are tracked as abandoned until that worker returns.
  5. Metrics expose active work, queue depth, cumulative abandonment, currently-active abandonment, and average queue wait time.

OffloadPool does not provide CPU isolation, CPU parallelism guarantees, or fairness between I/O and CPU-heavy work classes. Applications that need resource isolation may use named pools via Runtime#pool.

Examples:

Submitting synchronous work

op = runtime.offload.submit(timeout: 30) { expensive_call }
result = op.blocking_wait   # blocks the calling thread until done

With cancellation

token = Phronomy::Concurrency::CancellationToken.timeout_after(60)
op = pool.submit(timeout: 30, cancellation_token: token) { expensive_call }
result = op.blocking_wait

Defined Under Namespace

Classes: PendingOperation

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(pool_size: 10, queue_size: 100, name: nil, logger: nil, timer_queue_provider: nil) ⇒ OffloadPool

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of OffloadPool.

Parameters:

  • pool_size (Integer) (defaults to: 10) —

    maximum number of worker threads

  • queue_size (Integer) (defaults to: 100) —

    maximum pending operations waiting for a worker

  • name (String, Symbol, nil) (defaults to: nil) —

    optional pool name used in thread labels

  • logger (Logger, nil) (defaults to: nil) —

    optional logger for warnings

  • timer_queue_provider (#call, nil) (defaults to: nil) —

    returns a TimerQueue-compatible object. Required when submit(timeout:) or a monotonic-deadline cancellation token is used.



400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 400

def initialize(
  pool_size: 10,
  queue_size: 100,
  name: nil,
  logger: nil,
  timer_queue_provider: nil
)
  @pool_size = pool_size
  @queue_size = queue_size
  @name = name
  @logger = logger
  @timer_queue_provider = timer_queue_provider
  @queue = SizedQueue.new(queue_size)
  @active_count = 0
  @abandoned_count = 0
  @running_operation_ids = {}
  @abandoned_active_operation_ids = {}
  @total_wait_ns = 0
  @completed_count = 0
  @mutex = Mutex.new
  @shutdown = false
  @workers = Array.new(pool_size) { |i| spawn_worker(i) }
end

Instance Attribute Details

#name ⇒ String, ... (readonly)

Returns pool name used in thread labels.

Returns:

  • (String, Symbol, nil) —

    pool name used in thread labels



631
632
633
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 631

def name
  @name
end

#pool_size ⇒ Integer (readonly)

Returns configured maximum number of worker threads.

Returns:

  • (Integer) —

    configured maximum number of worker threads



625
626
627
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 625

def pool_size
  @pool_size
end

#queue_size ⇒ Integer (readonly)

Returns configured maximum queue depth.

Returns:

  • (Integer) —

    configured maximum queue depth



628
629
630
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 628

def queue_size
  @queue_size
end

Instance Method Details

#abandoned_active_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns number of abandoned operations that still occupy worker capacity at this instant.

Returns:

  • (Integer) —

    number of abandoned operations that still occupy worker capacity at this instant



608
609
610
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 608

def abandoned_active_count
  @mutex.synchronize { @abandoned_active_operation_ids.size }
end

#abandoned_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns cumulative number of operations whose caller-facing timeout or cancellation settled after worker execution had started.

Returns:

  • (Integer) —

    cumulative number of operations whose caller-facing timeout or cancellation settled after worker execution had started



601
602
603
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 601

def abandoned_count
  @mutex.synchronize { @abandoned_count }
end

#active_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns number of operations currently executing on workers.

Returns:

  • (Integer) —

    number of operations currently executing on workers



588
589
590
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 588

def active_count
  @mutex.synchronize { @active_count }
end

#average_wait_seconds ⇒ Float

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Average time (in seconds) that completed or skipped operations spent in the queue waiting for a worker. Returns 0.0 when none have been processed yet.

Returns:

  • (Float) 


616
617
618
619
620
621
622
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 616

def average_wait_seconds
  @mutex.synchronize do
    return 0.0 if @completed_count.zero?

    @total_wait_ns / @completed_count.to_f / 1_000_000_000.0
  end
end

#queue_depth ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns number of operations waiting in the queue.

Returns:

  • (Integer) —

    number of operations waiting in the queue



594
595
596
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 594

def queue_depth
  @queue.size
end

#shutdown(drain_timeout: 30) ⇒ self

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Gracefully drains the pool and terminates all worker threads. Waits up to drain_timeout seconds for in-flight operations to finish.

Closing the underlying SizedQueue signals workers to exit after draining remaining items, without blocking on a full-queue push.

Parameters:

  • drain_timeout (Numeric) (defaults to: 30) —

    seconds to wait for workers to finish

Returns:

  • (self) 


577
578
579
580
581
582
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 577

def shutdown(drain_timeout: 30)
  @shutdown = true
  @queue.close
  @workers.each { |thread| thread.join(drain_timeout) }
  self
end

#submit(timeout: nil, cancellation_token: nil, on_full: :wait, full_timeout: nil) { ... } ⇒ PendingOperation

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits synchronous off-EventLoop work to the pool. Returns a PendingOperation immediately after queue admission; the block runs on a worker thread. Do not submit logical waits (for example waiting for a child Agent Task) merely to make them asynchronous; those belong to FSMSession/EventLoop completion events.

A submit-time timeout is an operation-wide deadline measured from the start of this method, including queue wait. The timer settles the PendingOperation and notifies on_complete without forcibly interrupting a running worker. If the deadline fires before worker execution starts, the block is skipped. If it fires after execution starts, the operation is marked abandoned and the eventual worker result is discarded.

The submit cancellation_token is also operation-wide. Explicit cancellation settles the PendingOperation immediately. A token with a monotonic deadline is attached to the Runtime timer queue so deadline expiry becomes explicit cancellation without adding a polling Thread. Cancellation before execution skips the block; cancellation after execution starts abandons only the caller-facing result and never uses Thread#raise.

Synchronous queue admission may delay return from this method when on_full: :wait is used. EventLoop-owned framework paths therefore submit with on_full: :raise and handle backpressure asynchronously.

Parameters:

  • timeout (Numeric, nil) (defaults to: nil) —

    operation-wide deadline in seconds

  • cancellation_token (CancellationToken, nil) (defaults to: nil) —

    operation-wide token

  • on_full (Symbol) (defaults to: :wait) —

    :wait, :raise, or :timeout

  • full_timeout (Numeric, nil) (defaults to: nil) —

    queue-admission timeout for on_full: :timeout

Yields:

  • block containing synchronous work

Returns:

Raises:



460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
 
# File 'lib/phronomy/engine/concurrency/offload_pool.rb', line 460

def submit(
  timeout: nil,
  cancellation_token: nil,
  on_full: :wait,
  full_timeout: nil,
  &block
)
  raise Phronomy::PoolShutdownError, "pool has been shut down" if @shutdown

  submitted_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  already_cancelled = cancellation_token&.cancelled? || false
  cancellation_remaining = if already_cancelled
    nil
  else
    cancellation_token&.remaining_monotonic_seconds
  end

  needs_timer = !timeout.nil? ||
    (!cancellation_remaining.nil? && cancellation_remaining > 0)
  timer_queue = @timer_queue_provider&.call if needs_timer
  if needs_timer && !timer_queue
    raise Phronomy::ConfigurationError,
      "timer_queue is required when submit timeout or cancellation deadline is specified"
  end

  op = PendingOperation.new(
    block,
    timeout: timeout,
    cancellation_token: cancellation_token,
    submitted_at: submitted_at,
    on_abandoned: method(:record_abandoned)
  )

  # on_cancel only reacts to explicit cancel!, whereas cancelled? also covers a
  # monotonic deadline. Promote an already-expired deadline immediately.
  if already_cancelled
    cancellation_token.cancel!
    return op
  end

  begin
    if timeout
      elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - submitted_at
      remaining = timeout.to_f - elapsed
      if remaining <= 0
        op.fire_timeout!
        return op
      end

      # Arm before queue admission so the deadline includes time spent waiting
      # for a queue slot.
      timer_queue.schedule(seconds: remaining) { op.fire_timeout! }
    end

    if cancellation_remaining
      # Re-read after timeout setup so the scheduled delay reflects setup time.
      remaining = cancellation_token.remaining_monotonic_seconds
      if remaining <= 0
        cancellation_token.cancel!
        return op
      end
      timer_queue.schedule(seconds: remaining) { cancellation_token.cancel! }
    end

    # Cancellation/timeout can race with timer registration. Do not enqueue
    # already-settled work when the race is observable here.
    return op if op.done?

    case on_full
    when :raise
      begin
        @queue.push(op, true)
      rescue ThreadError
        raise Phronomy::BackpressureError,
          "OffloadPool queue is full (depth: #{@queue_size})"
      end
    when :timeout
      deadline = full_timeout ?
        (Process.clock_gettime(Process::CLOCK_MONOTONIC) + full_timeout) :
        nil
      loop do
        return op if op.done?

        @queue.push(op, true)
        break
      rescue ThreadError
        if deadline &&
            Process.clock_gettime(Process::CLOCK_MONOTONIC) >= deadline
          raise Phronomy::TimeoutError,
            "timed out waiting for a free slot in OffloadPool"
        end
        sleep(0.005)
      end
    else # :wait (default)
      @queue.push(op)
    end
  rescue ClosedQueueError => e
    # Shutdown raced with this submit — preserve the existing public error.
    op.fail_submission!(e)
    raise Phronomy::PoolShutdownError, "pool has been shut down"
  rescue => e
    op.fail_submission!(e)
    raise
  end

  op
end