Class: Woods::Cache::InflightEntry Private

Inherits:
Object
  • Object
show all
Defined in:
lib/woods/cache/cache_middleware.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Per-text in-flight entry used for single-flight coordination in CachedEmbeddingProvider#embed_batch. When thread A is already fetching an embedding for text T, thread B’s miss for T attaches to A’s entry and waits on its condition variable rather than issuing a parallel provider call. See issue #88.

Instance Method Summary collapse

Constructor Details

#initialize ⇒ InflightEntry

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of InflightEntry.



28
29
30
31
32
33
34
35
 
# File 'lib/woods/cache/cache_middleware.rb', line 28

def initialize
  @mutex = Mutex.new
  @cond = ConditionVariable.new
  @done = false
  @value = nil
  @error = nil
  @waiter_count = 0
end

Instance Method Details

#await ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Block until #fulfill or #reject is called, then return the value (or re-raise the error) to the waiting thread. ‘@waiter_count` is bumped under the mutex so tests can deterministically wait for “N threads have attached to this entry” instead of polling coarse Thread#status values.

Raises:

  • (@error) 


66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/woods/cache/cache_middleware.rb', line 66

def await
  @mutex.synchronize do
    @waiter_count += 1
    begin
      @cond.wait(@mutex) until @done
    ensure
      @waiter_count -= 1
    end
  end
  raise @error if @error

  @value
end

#fulfill(value) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Publish the computed value and wake every waiter. Idempotent — a second call (e.g. from an ‘ensure` that rejects unfulfilled entries) is a no-op so the hardening in CachedEmbeddingProvider#fetch_and_fulfill is safe.



40
41
42
43
44
45
46
47
48
 
# File 'lib/woods/cache/cache_middleware.rb', line 40

def fulfill(value)
  @mutex.synchronize do
    return if @done

    @value = value
    @done = true
    @cond.broadcast
  end
end

#reject(error) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Publish an exception so waiters fail fast instead of blocking forever. Idempotent — see #fulfill.



52
53
54
55
56
57
58
59
60
 
# File 'lib/woods/cache/cache_middleware.rb', line 52

def reject(error)
  @mutex.synchronize do
    return if @done

    @error = error
    @done = true
    @cond.broadcast
  end
end

#waiter_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Number of threads currently blocked in #await. Thread-safe observation used primarily by concurrent specs to synchronize without relying on ‘Thread#status` (which can transiently report ’sleep’ on unrelated mutex contention — see issue #94 CI flake on MRI 3.1/3.2).

Returns:

  • (Integer) 


86
87
88
 
# File 'lib/woods/cache/cache_middleware.rb', line 86

def waiter_count
  @mutex.synchronize { @waiter_count }
end