Class: Nonnative::FaultInjectionProxy

Inherits:
Proxy
  • Object
show all
Defined in:
lib/nonnative/fault_injection_proxy.rb

Overview

Fault-injection proxy for TCP services.

This proxy accepts incoming TCP connections and forwards traffic to the configured upstream (service.proxy.host / service.proxy.port) via a socket-pair implementation. It can also inject failures to help validate client resilience.

This class exposes a small public control surface for tests:

  • #close_all: close connections immediately on accept
  • #reset_peer: reset connections immediately on accept (TCP RST rather than a graceful close)
  • #delay: delay reads by a configured duration (default: 2 seconds)
  • #timeout: accept connections and keep them silent until clients time out
  • #invalid_data: forward requests unchanged and mutate upstream responses before they reach clients
  • #bandwidth: throttle forwarded throughput to a configured rate (KB/s)
  • #limit_data: forward a configured number of response bytes, then gracefully close
  • #slicer: forward responses to the client in small writes to force multi-recv reassembly
  • #flaky: fail a configurable fraction of new connections, forwarding the rest normally
  • #reset: return to healthy pass-through behavior

State changes terminate any active connections so new connections observe the new behavior.

Wiring

When enabled, your test/client should connect to the service host and port (the proxy endpoint), and the proxy will forward traffic to the upstream target exposed by #host:#port.

Configuration

The proxy is configured via the service's proxy hash:

  • kind: "fault_injection"
  • host / port: upstream target behind the proxy (exposed via #host/#port)
  • log: file path used by this proxy’s internal logger
  • wait: sleep interval (seconds) applied after state changes
  • options:
    • delay: delay duration in seconds used by #delay
    • jitter: optional random offset (seconds) added in -jitter..jitter to each delay (a negative value uses its magnitude), so clients see variable latency instead of a flat value
    • rate: positive throughput limit in KB/s used by #bandwidth; absent or non-positive values forward at full speed
    • bytes: positive response byte limit used by #limit_data; absent or non-positive values use pass-through behavior
    • slice_size: positive response slice size (bytes) used by #slicer; absent or non-positive values use pass-through behavior
    • slice_delay: optional delay (seconds) between slices used by #slicer
    • probability: connection failure fraction (0.0-1.0) used by #flaky; absent or non-positive values use pass-through behavior

See Also:

Defined Under Namespace

Classes: Connection

Constant Summary collapse

STOP_DRAIN_TIMEOUT =

Used both to flush the accept-queue barrier and as the aggregate deadline for draining active worker threads during stop.

1

Instance Method Summary collapse

Constructor Details

#initialize(service) ⇒ FaultInjectionProxy

Returns a new instance of FaultInjectionProxy.

Parameters:



74
75
76
77
78
79
80
81
82
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 74

def initialize(service)
  @connections = Concurrent::Hash.new
  @logger = Logger.new(service.proxy.log)
  @mutex = Mutex.new
  @state = :none
  @stopping = false

  super
end

Instance Method Details

#bandwidth ⇒ void

This method returns an undefined value.

Throttles forwarded throughput to a configured rate.

The rate is controlled by service.proxy.options[:rate] (kilobytes per second); when it is absent or not positive the connection forwards at full speed.



173
174
175
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 173

def bandwidth
  apply_state :bandwidth
end

#close_all ⇒ void

This method returns an undefined value.

Forces new connections to be closed immediately.



126
127
128
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 126

def close_all
  apply_state :close_all
end

#delay ⇒ void

This method returns an undefined value.

Delays reads before forwarding.

The delay duration is controlled by service.proxy.options[:delay] and defaults to 2 seconds.



145
146
147
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 145

def delay
  apply_state :delay
end

#flaky ⇒ void

This method returns an undefined value.

Fails a configurable fraction of new connections while forwarding the rest normally.

The fraction is controlled by service.proxy.options[:probability] (0.0-1.0); absent or non-positive values behave like pass-through, and 1.0 fails every connection. Because each connection decides independently, clients that retry/reconnect can observe both failures and successes while this state stays active.



209
210
211
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 209

def flaky
  apply_state :flaky
end

#host ⇒ String

Returns the upstream host behind this proxy.

Returns:

  • (String) 


223
224
225
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 223

def host
  service.proxy.host
end

#invalid_data ⇒ void

This method returns an undefined value.

Mutates upstream responses while forwarding client requests unchanged.



163
164
165
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 163

def invalid_data
  apply_state :invalid_data
end

#limit_data ⇒ void

This method returns an undefined value.

Truncates the upstream byte stream after a configured number of bytes.

Client requests are forwarded unchanged. The response byte limit is read from service.proxy.options[:bytes]; when it is absent or not positive, the connection forwards at full speed without truncation.



184
185
186
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 184

def limit_data
  apply_state :limit_data
end

#port ⇒ Integer

Returns the upstream port behind this proxy.

Returns:

  • (Integer) 


230
231
232
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 230

def port
  service.proxy.port
end

#reset ⇒ void

This method returns an undefined value.

Resets the proxy back to healthy pass-through behavior.



216
217
218
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 216

def reset
  apply_state :none
end

#reset_peer ⇒ void

This method returns an undefined value.

Forces new connections to be reset immediately.

Unlike #close_all, which closes the socket gracefully (FIN), this closes the accepted socket with a zero linger timeout so clients observe a TCP reset (Errno::ECONNRESET).



136
137
138
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 136

def reset_peer
  apply_state :reset_peer
end

#slicer ⇒ void

This method returns an undefined value.

Fragments upstream responses into small writes before forwarding to the client.

Client requests are forwarded unchanged. Each response is split into service.proxy.options[:slice_size]-byte writes, optionally separated by service.proxy.options[:slice_delay] seconds, so a client's recv returns a strict prefix of the response rather than the whole message. When slice_size is absent or not positive, the connection forwards at full speed without slicing.



197
198
199
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 197

def slicer
  apply_state :slicer
end

#start ⇒ void

This method returns an undefined value.

Starts the proxy accept loop in a background thread.

This binds a TCP server on the service host and port. Clients connect to that service endpoint, while upstream traffic is forwarded to #host:#port.



90
91
92
93
94
95
96
97
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 90

def start
  mutex.synchronize { @stopping = false }
  logger
  @tcp_server = ::TCPServer.new(service.host, service.port)
  @thread = Thread.new { perform_start }

  Nonnative.logger.info "started with host '#{service.host}' and port '#{service.port}' for proxy 'fault_injection'"
end

#stop ⇒ void

This method returns an undefined value.

Stops the proxy, closes active connections, and closes its listening socket.



102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 102

def stop
  server = tcp_server
  mark_stopping
  drain_workers(close_connections)
  close_queued_connections(server)
  server&.close

  listener_thread = thread
  # Closing the server is meant to wake the blocked `accept`, but a concurrently blocked
  # `accept` is not reliably interrupted by `close` on every platform. Kill the listener so
  # `stop` cannot hang joining an accept loop that never woke (a no-op once it has exited).
  listener_thread&.kill
  listener_thread&.join

  @tcp_server = nil
  @thread = nil
  close_logger

  Nonnative.logger.info "stopped with host '#{service.host}' and port '#{service.port}' for proxy 'fault_injection'"
end

#timeout ⇒ void

This method returns an undefined value.

Accepts connections and stalls without forwarding bytes.

This simulates a dependency that accepts a TCP connection but leaves clients waiting until their own read timeout fires. The proxy keeps the connection silent until reset or stop closes active connections.



156
157
158
 
# File 'lib/nonnative/fault_injection_proxy.rb', line 156

def timeout
  apply_state :timeout
end