Class: Nonnative::FaultInjectionProxy

Inherits:
Proxy
  • Object
show all
Defined in:
lib/nonnative/fault_injection_proxy.rb

Overview

Fault-injection proxy for TCP services.

This proxy accepts incoming TCP connections and forwards traffic to the configured upstream (service.proxy.host / service.proxy.port) via a socket-pair implementation. It can also inject failures to help validate client resilience.

This class exposes a small public control surface for tests:

  • #close_all: close connections immediately on accept
  • #reset_peer: reset connections immediately on accept (TCP RST rather than a graceful close)
  • #delay: delay reads by a configured duration (default: 2 seconds)
  • #timeout: accept connections and keep them silent until clients time out
  • #invalid_data: forward requests unchanged and mutate upstream responses before they reach clients
  • #bandwidth: throttle forwarded throughput to a configured rate (KB/s)
  • #limit_data: forward a configured number of response bytes, then gracefully close
  • #slicer: forward responses to the client in small writes to force multi-recv reassembly
  • #flaky: fail a configurable fraction of new connections, forwarding the rest normally
  • #reset: return to healthy pass-through behavior

State changes terminate any active connections so new connections observe the new behavior.

Wiring

When enabled, your test/client should connect to the service host and port (the proxy endpoint), and the proxy will forward traffic to the upstream target exposed by #host:#port.

Configuration

The proxy is configured via the service's proxy hash:

  • kind: "fault_injection"
  • host / port: upstream target behind the proxy (exposed via #host/#port)
  • log: file path used by this proxy’s internal logger
  • wait: sleep interval (seconds) applied after state changes
  • options:
    • delay: delay duration in seconds used by #delay
    • jitter: optional random offset (seconds) added in -jitter..jitter to each delay (a negative value uses its magnitude), so clients see variable latency instead of a flat value
    • rate: positive throughput limit in KB/s used by #bandwidth; absent or non-positive values forward at full speed
    • bytes: positive response byte limit used by #limit_data; absent or non-positive values use pass-through behavior
    • slice_size: positive response slice size (bytes) used by #slicer; absent or non-positive values use pass-through behavior
    • slice_delay: optional delay (seconds) between slices used by #slicer
    • probability: connection failure fraction (0.0-1.0) used by #flaky; absent or non-positive values use pass-through behavior

See Also:

Defined Under Namespace

Classes: Connection

Constant Summary collapse

STOP_DRAIN_TIMEOUT = 
1

Instance Method Summary collapse

Constructor Details

#initialize(service) ⇒ FaultInjectionProxy

Returns a new instance of FaultInjectionProxy.

Parameters:



73
74
75
76
77
78
79
80
81
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 73

def initialize(service)
  @connections = Concurrent::Hash.new
  @logger = Logger.new(service.proxy.log)
  @mutex = Mutex.new
  @state = :none
  @stopping = false

  super
end

Instance Method Details

#bandwidth ⇒ void

This method returns an undefined value.

Throttles forwarded throughput to a configured rate.

The rate is controlled by service.proxy.options[:rate] (kilobytes per second); when it is absent or not positive the connection forwards at full speed.



170
171
172
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 170

def bandwidth
  apply_state :bandwidth
end

#close_all ⇒ void

This method returns an undefined value.

Forces new connections to be closed immediately.



123
124
125
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 123

def close_all
  apply_state :close_all
end

#delay ⇒ void

This method returns an undefined value.

Delays reads before forwarding.

The delay duration is controlled by service.proxy.options[:delay] and defaults to 2 seconds.



142
143
144
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 142

def delay
  apply_state :delay
end

#flaky ⇒ void

This method returns an undefined value.

Fails a configurable fraction of new connections while forwarding the rest normally.

The fraction is controlled by service.proxy.options[:probability] (0.0-1.0); absent or non-positive values behave like pass-through, and 1.0 fails every connection. Because each connection decides independently, clients that retry/reconnect can observe both failures and successes while this state stays active.



206
207
208
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 206

def flaky
  apply_state :flaky
end

#host ⇒ String

Returns the upstream host behind this proxy.

Returns:

  • (String) 


220
221
222
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 220

def host
  service.proxy.host
end

#invalid_data ⇒ void

This method returns an undefined value.

Mutates upstream responses while forwarding client requests unchanged.



160
161
162
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 160

def invalid_data
  apply_state :invalid_data
end

#limit_data ⇒ void

This method returns an undefined value.

Truncates the upstream byte stream after a configured number of bytes.

Client requests are forwarded unchanged. The response byte limit is read from service.proxy.options[:bytes]; when it is absent or not positive, the connection forwards at full speed without truncation.



181
182
183
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 181

def limit_data
  apply_state :limit_data
end

#port ⇒ Integer

Returns the upstream port behind this proxy.

Returns:

  • (Integer) 


227
228
229
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 227

def port
  service.proxy.port
end

#reset ⇒ void

This method returns an undefined value.

Resets the proxy back to healthy pass-through behavior.



213
214
215
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 213

def reset
  apply_state :none
end

#reset_peer ⇒ void

This method returns an undefined value.

Forces new connections to be reset immediately.

Unlike #close_all, which closes the socket gracefully (FIN), this closes the accepted socket with a zero linger timeout so clients observe a TCP reset (Errno::ECONNRESET).



133
134
135
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 133

def reset_peer
  apply_state :reset_peer
end

#slicer ⇒ void

This method returns an undefined value.

Fragments upstream responses into small writes before forwarding to the client.

Client requests are forwarded unchanged. Each response is split into service.proxy.options[:slice_size]-byte writes, optionally separated by service.proxy.options[:slice_delay] seconds, so a client's recv returns a strict prefix of the response rather than the whole message. When slice_size is absent or not positive, the connection forwards at full speed without slicing.



194
195
196
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 194

def slicer
  apply_state :slicer
end

#start ⇒ void

This method returns an undefined value.

Starts the proxy accept loop in a background thread.

This binds a TCP server on the service host and port. Clients connect to that service endpoint, while upstream traffic is forwarded to #host:#port.



89
90
91
92
93
94
95
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 89

def start
  mutex.synchronize { @stopping = false }
  @tcp_server = ::TCPServer.new(service.host, service.port)
  @thread = Thread.new { perform_start }

  Nonnative.logger.info "started with host '#{service.host}' and port '#{service.port}' for proxy 'fault_injection'"
end

#stop ⇒ void

This method returns an undefined value.

Stops the proxy, closes active connections, and closes its listening socket.



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 100

def stop
  server = tcp_server
  mark_stopping
  close_connections
  close_queued_connections(server)
  server&.close

  listener_thread = thread
  # Closing the server is meant to wake the blocked `accept`, but a concurrently blocked
  # `accept` is not reliably interrupted by `close` on every platform. Kill the listener so
  # `stop` cannot hang joining an accept loop that never woke (a no-op once it has exited).
  listener_thread&.kill
  listener_thread&.join

  @tcp_server = nil
  @thread = nil

  Nonnative.logger.info "stopped with host '#{service.host}' and port '#{service.port}' for proxy 'fault_injection'"
end

#timeout ⇒ void

This method returns an undefined value.

Accepts connections and stalls without forwarding bytes.

This simulates a dependency that accepts a TCP connection but leaves clients waiting until their own read timeout fires. The proxy keeps the connection silent until reset or stop closes active connections.



153
154
155
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 153

def timeout
  apply_state :timeout
end