Class: Faulty::Storage::FaultTolerantProxy

Inherits:
Object
  • Object
show all
Extended by:
Forwardable
Defined in:
lib/faulty/storage/fault_tolerant_proxy.rb

Overview

A wrapper for storage backends that may raise errors

Faulty#initialize automatically wraps all non-fault-tolerant storage backends with this class.

If the storage backend raises a StandardError, it will be captured and sent to the notifier.

The overall design preference is to keep protected code paths running when the storage backend is degraded, even when that means losing circuit-breaker protections that the storage normally provides: #status returns a stub closed status (so Circuit#run proceeds), #reserve returns true (so half-open test runs proceed), and the write paths (#open, #reopen, #close, #entry) return false to safe-deny the recorded transition without failing the in-flight call. The trade-off is that a correlated outage of the storage backend and the upstream protected by the circuit will let the fleet converge on the upstream — but that fleet would converge anyway via the stub-closed status path, so individual write methods don't make it worse.

Defined Under Namespace

Classes: Options

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(storage, **options) {|Options| ... } ⇒ FaultTolerantProxy

Returns a new instance of FaultTolerantProxy.

Parameters:

Yields:

  • (Options) —

    For setting options in a block



47
48
49
50
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 47

def initialize(storage, **options, &)
  @storage = storage
  @options = Options.new(options, &)
end

Instance Attribute Details

#options ⇒ Object (readonly)

Returns the value of attribute options.



28
29
30
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 28

def options
  @options
end

Class Method Details

.wrap(storage) ⇒ Storage::Interface

Wrap a storage backend in a FaultTolerantProxy unless it's already fault tolerant

Parameters:

Returns:



57
58
59
60
61
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 57

def self.wrap(storage, ...)
  return storage if storage.fault_tolerant?

  new(storage, ...)
end

Instance Method Details

#clear ⇒ Object

Clear is not called in normal operation, so it doesn't capture errors

See Also:



104
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 104

def_delegators :@storage, :lock, :unlock, :reset, :history, :list, :clear

#close(circuit) ⇒ Boolean

Safely mark a circuit as closed

Returns:

  • (Boolean) —

    True if the circuit transitioned from open to closed

See Also:



171
172
173
174
175
176
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 171

def close(circuit)
  @storage.close(circuit)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :close, error: e)
  false
end

#entry(circuit, time, success, status) ⇒ Status?

Add a history entry safely

Parameters:

  • circuit (Circuit) —

    The circuit that ran

  • time (Float) —

    The unix timestamp for the run, from Faulty.current_time

  • success (Boolean) —

    True if the run succeeded

  • status (Status, nil) —

    The previous status. If given, this method must return an updated status object from the new entry data.

Returns:

  • (Status, nil) —

    If status is not nil, the updated status object.

See Also:



135
136
137
138
139
140
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 135

def entry(circuit, time, success, status)
  @storage.entry(circuit, time, success, status)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :entry, error: e)
  stub_status(circuit) if status
end

#fault_tolerant? ⇒ true

This cache makes any storage fault tolerant, so this is always true

Returns:

  • (true) 


212
213
214
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 212

def fault_tolerant?
  true
end

#get_options(circuit) ⇒ Hash

Get circuit options safely

Returns:

  • (Hash) —

    A hash of the options stored by #set_options. The keys must be symbols.

See Also:



111
112
113
114
115
116
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 111

def get_options(circuit)
  @storage.get_options(circuit)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :get_options, error: e)
  nil
end

#history(circuit) ⇒ Object

History is not called in normal operation, so it doesn't capture errors

See Also:



104
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 104

def_delegators :@storage, :lock, :unlock, :reset, :history, :list, :clear

#list ⇒ Object

List is not called in normal operation, so it doesn't capture errors

See Also:



104
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 104

def_delegators :@storage, :lock, :unlock, :reset, :history, :list, :clear

#lock(circuit, state) ⇒ Object

Lock is not called in normal operation, so it doesn't capture errors

See Also:



104
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 104

def_delegators :@storage, :lock, :unlock, :reset, :history, :list, :clear

#open(circuit, opened_at) ⇒ Boolean

Safely mark a circuit as open

Parameters:

  • circuit (Circuit) —

    The circuit to open

  • opened_at (Float) —

    The timestamp the circuit was opened at, from Faulty.current_time

Returns:

  • (Boolean) —

    True if the circuit transitioned from closed to open

See Also:



147
148
149
150
151
152
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 147

def open(circuit, opened_at)
  @storage.open(circuit, opened_at)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :open, error: e)
  false
end

#reopen(circuit, opened_at, previous_opened_at) ⇒ Boolean

Safely mark a circuit as reopened

Parameters:

  • circuit (Circuit) —

    The circuit to reopen

  • opened_at (Float) —

    The timestamp the circuit was opened at, from Faulty.current_time

  • previous_opened_at (Float) —

    The last known value of opened_at. Can be used to compare-and-set. Always non-nil — Circuit#failure! only enters the reopen branch when status.half_open? is true, which requires non-nil opened_at. Unlike previous_reserved_at on #reserve, there is no legitimate "no prior value" call path to reopen, so backends may treat this parameter as required and are not expected to handle nil.

Returns:

  • (Boolean) —

    True if the opened_at time was updated

See Also:



159
160
161
162
163
164
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 159

def reopen(circuit, opened_at, previous_opened_at)
  @storage.reopen(circuit, opened_at, previous_opened_at)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :reopen, error: e)
  false
end

#reserve(circuit, reserved_at, previous_reserved_at) ⇒ Boolean

Safely reserve execution of a circuit

Returns true on storage error so half-open test runs proceed when the backend is degraded. See the class-level docs for the gem's fail-open trade-off.

Parameters:

  • circuit (Circuit) —

    The circuit to reserve

  • reserved_at (Float) —

    The timestamp of this reservation, from Faulty.current_time

  • previous_reserved_at (Float, nil) —

    The last known value of reserved_at, or nil for the first reservation in a new open cycle. Can be used to compare-and-set.

Returns:

  • (Boolean) —

    True if the caller may proceed with the half-open test run; false if another caller already holds the reservation.

See Also:



202
203
204
205
206
207
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 202

def reserve(circuit, reserved_at, previous_reserved_at)
  @storage.reserve(circuit, reserved_at, previous_reserved_at)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :reserve, error: e)
  true
end

#reset(circuit) ⇒ Object

Reset is not called in normal operation, so it doesn't capture errors

See Also:



104
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 104

def_delegators :@storage, :lock, :unlock, :reset, :history, :list, :clear

#set_options(circuit, stored_options) ⇒ void

This method returns an undefined value.

Set circuit options safely

Parameters:

  • circuit (Circuit) —

    The circuit to set options for

  • stored_options (Hash<Symbol, Object>) —

    A hash of symbol option names to circuit options. These option values are guranteed to be primive values.

See Also:



123
124
125
126
127
128
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 123

def set_options(circuit, stored_options)
  @storage.set_options(circuit, stored_options)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :set_options, error: e)
  nil
end

#status(circuit) ⇒ Status

Safely get the status of a circuit

If the backend is unavailable, this returns a stub status that indicates that the circuit is closed.

Parameters:

  • circuit (Circuit) —

    The circuit to get status for

Returns:

  • (Status) —

    The current status

See Also:



186
187
188
189
190
191
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 186

def status(circuit)
  @storage.status(circuit)
rescue StandardError => e
  options.notifier.notify(:storage_failure, circuit: circuit, action: :status, error: e)
  stub_status(circuit)
end

#unlock(circuit) ⇒ Object

Unlock is not called in normal operation, so it doesn't capture errors

See Also:



104
 
# File 'lib/faulty/storage/fault_tolerant_proxy.rb', line 104

def_delegators :@storage, :lock, :unlock, :reset, :history, :list, :clear