CircleCI codecov Gem Version Stability: Active

🧪 Nonnative

Nonnative is a Ruby-first harness for end-to-end testing of systems implemented in other languages.

It helps you:

  • start OS processes (e.g. your Go/Java/Rust service binary),
  • start in-process Ruby servers (e.g. small HTTP/TCP/gRPC fakes for dependencies),
  • optionally start proxies in front of processes/servers/services for fault-injection,
  • wait for readiness/shutdown using TCP port checks.

Once started, you can test however you like (TCP, HTTP, gRPC, etc).

📦 Installation

[!IMPORTANT] Nonnative currently supports Ruby >= 4.0.0 and < 5.0.0.

Add this line to your application's Gemfile:

gem 'nonnative'

And then execute:

bundle

Or install it yourself as:

gem install nonnative

🚀 Usage

Nonnative is configured via Nonnative.configure (programmatic) or config.load_file(...) (YAML). YAML configuration is loaded as data only: ERB is not evaluated and arbitrary Ruby objects are not deserialized.

[!CAUTION] Treat YAML configuration as plain data. ERB is not evaluated and arbitrary Ruby object tags are rejected.

High-level configuration fields:

  • version: configuration version (example: "1.0").
  • name: logical system name (used by Nonnative.observability for /<name>/healthz, etc).
  • url: base URL for observability queries (example: http://localhost:4567).
  • log: path for the Nonnative logger output.
  • processes: child processes to spawn.
  • servers: in-process Ruby servers started in threads.
  • services: external dependencies (proxy-only; no process/thread started by Nonnative).

Common runner fields:

  • name: runner name used for lookup.
  • host/ports: client-facing address. host defaults to 127.0.0.1. For processes and servers, this address is also used for readiness/shutdown port checks. When a fault_injection proxy is enabled, clients should hit the first configured port.

Process/server fields:

  • timeout: max time (seconds) for readiness/shutdown port checks.
  • wait: small sleep (seconds) between lifecycle steps.
  • log: per-runner log file used by process output redirection or server implementations.

For fault_injection, the nested proxy.host/proxy.port describe the upstream target behind the proxy. Nested proxy.host also defaults to 127.0.0.1. In-process server implementations typically bind there via proxy.host / proxy.port.

[!IMPORTANT] When a proxy is enabled, tests and clients connect to the runner host and first configured ports entry; the nested proxy.host/proxy.port is the upstream target behind the proxy.

Nonnative readiness and shutdown checks are TCP-only. Configure ports that are dedicated to the test run; if another process is already listening on the same host/ports endpoint, results are undefined.

[!WARNING] Readiness and shutdown checks only prove that a TCP port opened or closed. They do not verify HTTP status, gRPC health, schema readiness, migrations, or application-specific health.

Start and stop Nonnative around the test scope that should own the configured runners:

require 'nonnative'

Nonnative.configure do |config|
  config.load_file('configuration.yml')
end

Nonnative.start
# run tests...
Nonnative.stop

Nonnative.start starts services first, then servers and processes. Nonnative.stop stops processes and servers first, then services. If startup fails, Nonnative rolls back runners that already started and raises Nonnative::StartError; shutdown failures raise Nonnative::StopError.

[!NOTE] Nonnative.clear clears memoized configuration, logger, observability client, and pool. Use it before reconfiguring Nonnative in the same Ruby process.

📈 Observability

Nonnative.observability is an HTTP client for common service endpoints under the configured name and url:

  • health(...): calls /<name>/healthz.
  • liveness(...): calls /<name>/livez.
  • readiness(...): calls /<name>/readyz.
  • metrics(...): calls /<name>/metrics.

Each method accepts RestClient options such as headers, open_timeout, and read_timeout.

response = Nonnative.observability.health(
  headers: { content_type: :json, accept: :json },
  open_timeout: 2,
  read_timeout: 2
)

expect(response.code).to eq(200)

🔁 Lifecycle strategies (Cucumber integration)

Nonnative ships Cucumber hooks (when loaded) that support these tags/strategies:

  • @startup: start before scenario; stop after scenario
  • @manual: stop after scenario (start is expected to be triggered manually in steps)
  • @clear: clears memoized configuration, logger, observability client, and pool before scenario
  • @reset: resets proxies after scenario

The repo’s own Cucumber suite also uses taxonomy tags to classify coverage:

  • @acceptance: end-to-end behavior across configured runners and clients
  • @contract: lower-level contract and lifecycle behavior
  • @proxy: proxy-specific behavior and failure injection
  • @config: coverage that exercises YAML/config loading
  • @service: scenarios centered on externally managed dependencies
  • @benchmark: benchmark-only scenarios run by make benchmarks
  • @slow: slower scenarios, currently used by benchmark coverage

make features excludes @benchmark, while make benchmarks runs only @benchmark.

Requiring nonnative is enough; the Cucumber hooks and step definitions are installed lazily once Cucumber’s Ruby DSL is ready.

If you want "start once per test run", require:

require 'nonnative/startup'

This calls Nonnative.start immediately and registers an at_exit stop.

⚙️ Processes

A process is some sort of command that you would run locally. Programmatic p.command values must be callables that return a shell string or an argv array. YAML command values can be scalars or lists and are wrapped internally. String commands preserve legacy shell semantics, while argv arrays avoid shell interpretation and are preferred for new configuration.

[!TIP] Prefer argv arrays for new process commands. Use shell strings only when you intentionally need shell parsing, expansion, or redirection.

Set it up programmatically:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.process do |p|
    p.name = 'start_1'
    p.command = -> { ['features/support/bin/start', '12_321'] }
    p.timeout = 5
    p.wait = 0.1
    p.ports = [12_321]
    p.log = '12_321.log'
    p.signal = 'INT' # Possible values are described in Signal.list.keys.
    p.environment = { # Pass environment variables to process.
      'TEST' => 'true'
    }
  end

  config.process do |p|
    p.name = 'start_2'
    p.command = -> { ['features/support/bin/start', '12_322'] }
    p.timeout = 0.5
    p.wait = 0.1
    p.ports = [12_322]
    p.log = '12_322.log'
  end
end

Set it up through configuration:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
processes:
  -
    name: start_1
    command:
      - features/support/bin/start
      - "12_321"
    timeout: 5
    wait: 1
    ports:
      - 12321
    log: 12_321.log
    signal: INT # Possible values are described in Signal.list.keys.
    environment: # Pass environment variables to process.
      TEST: true
  -
    name: start_2
    command:
      - features/support/bin/start
      - "12_322"
    timeout: 5
    wait: 1
    ports:
      - 12322
    log: 12_322.log

Then load the file with

require 'nonnative'

Nonnative.configure do |config|
  config.load_file('configuration.yml')
end

With cucumber you can also verify how much memory is used by the process:

Then the process 'start_1' should consume less than '25mb' of memory

🖥️ Servers

A server is an in-process Ruby fake or helper server that Nonnative starts in a thread. Use servers for dependencies that are easiest to model inside the test process, such as small TCP, HTTP, or gRPC fakes.

Define your server:

module Nonnative
  class TCPServer < Nonnative::Server
    def initialize(service)
      super

      @socket_server = ::TCPServer.new(proxy.host, proxy.port)
    end

    def perform_start
      loop do
        client_socket = socket_server.accept
        client_socket.puts 'Hello World!'
        client_socket.close
      end
    rescue StandardError
      socket_server.close
    end

    def perform_stop
      socket_server.close
    end

    private

    attr_reader :socket_server
  end
end

Set it up programmatically:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.server do |s|
    s.name = 'server_1'
    s.klass = Nonnative::TCPServer
    s.timeout = 1
    s.ports = [12_323]
    s.log = 'server_1.log'
  end

  config.server do |s|
    s.name = 'server_2'
    s.klass = Nonnative::TCPServer
    s.timeout = 1
    s.ports = [12_324]
    s.log = 'server_2.log'
  end
end

Set it up through configuration:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
servers:
  -
    name: server_1
    class: Nonnative::TCPServer
    timeout: 1
    ports:
      - 12323
    log: server_1.log
  -
    name: server_2
    class: Nonnative::TCPServer
    timeout: 1
    ports:
      - 12324
    log: server_2.log

Then load the file with:

require 'nonnative'

Nonnative.configure do |config|
  config.load_file('configuration.yml')
end

🌐 HTTP

Define your server:

module Nonnative
  module Features
    class Hello < Sinatra::Application
      get '/hello' do
        'Hello World!'
      end
    end

    class HTTPServer < Nonnative::HTTPServer
      def initialize(service)
        super(Sinatra.new(Hello), service)
      end
    end
  end
end

Set it up programmatically:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.server do |s|
    s.name = 'http_server_1'
    s.klass = Nonnative::Features::HTTPServer
    s.timeout = 1
    s.ports = [4567]
    s.log = 'http_server_1.log'
  end
end

Set it up through configuration:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
servers:
  -
    name: http_server_1
    class: Nonnative::Features::HTTPServer
    timeout: 1
    ports:
      - 4567
    log: http_server_1.log

Then load the file with:

require 'nonnative'

Nonnative.configure do |config|
  config.load_file('configuration.yml')
end
🔀 Proxy

The system allows you to define an HTTP proxy for external systems, e.g. api.github.com.

Define your server:

module Nonnative
  module Features
    class HTTPProxyServer < Nonnative::HTTPProxyServer
      def initialize(service)
        super('www.afalkowski.com', service)
      end
    end
  end
end

Set it up programmatically:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.server do |s|
    s.name = 'http_server_proxy'
    s.klass = Nonnative::Features::HTTPProxyServer
    s.timeout = 1
    s.ports = [4567]
    s.log = 'http_server_proxy.log'
  end
end

Set it up through configuration:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
servers:
  -
    name: http_server_proxy
    class: Nonnative::Features::HTTPProxyServer
    timeout: 1
    ports:
      - 4567
    log: http_server_proxy.log

Then load the file with:

require 'nonnative'

Nonnative.configure do |config|
  config.load_file('configuration.yml')
end

📡 gRPC

Define your server:

module Nonnative
  module Features
    class Greeter < GreeterService::Service
      def say_hello(request, _call)
        Nonnative::Features::SayHelloResponse.new(message: request.name.to_s)
      end
    end

    class GRPCServer < Nonnative::GRPCServer
      def initialize(service)
        super(Greeter.new, service)
      end
    end
  end
end

Set it up programmatically:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.server do |s|
    s.name = 'grpc_server_1'
    s.klass = Nonnative::Features::GRPCServer
    s.timeout = 1
    s.ports = [9002]
    s.log = 'grpc_server_1.log'
  end
end

Set it up through configuration:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
servers:
  -
    name: grpc_server_1
    class: Nonnative::Features::GRPCServer
    timeout: 1
    ports:
      - 9002
    log: grpc_server_1.log

Then load the file with:

require 'nonnative'

Nonnative.configure do |config|
  config.load_file('configuration.yml')
end

🧩 Services

A service is an external dependency to your system that you do not want Nonnative to start (no OS process, no Ruby thread). Services are primarily useful when paired with proxies, because they let you inject failures into dependencies that are managed elsewhere (e.g. a DB running in Docker).

Services do not get process lifecycle management or TCP readiness/shutdown checks from Nonnative. They only provide a named runner and optional proxy lifecycle for a dependency that another tool already manages.

Set it up programmatically:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.service do |s|
    s.name = 'postgres'
    s.host = '127.0.0.1'
    s.ports = [5432]
  end

  config.service do |s|
    s.name = 'redis'
    s.host = '127.0.0.1'
    s.ports = [6379]
  end
end

Set it up through configuration (YAML):

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
services:
  -
    name: postgres
    host: 127.0.0.1
    ports:
      - 5432
  -
    name: redis
    host: 127.0.0.1
    ports:
      - 6379

Then load the file with:

require 'nonnative'

Nonnative.configure do |config|
  config.load_file('configuration.yml')
end

🕸️ Proxies

These proxies can simulate different situations. Available proxy kinds are:

  • none (this is the default)
  • fault_injection

[!WARNING] Unknown proxy kinds fall back to none. If fault injection is not taking effect, check the kind spelling or register the custom kind before loading the configuration.

Custom proxy kinds can be registered through Nonnative.proxies:

Nonnative.proxies['custom'] = CustomProxy

For fault_injection, keep the runner host and first ports entry as the client-facing endpoint and use nested proxy.host/proxy.port for the upstream target behind the proxy.

⚙️ Process Proxies

Add this to an existing process configuration:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.process do |p|
    p.host = '127.0.0.1'
    p.ports = [20_000]

    p.proxy = {
      kind: 'fault_injection',
      host: '127.0.0.1',
      port: 12_321,
      log: 'proxy_server.log',
      wait: 1,
      options: {
        delay: 5
      }
    }
  end
end

YAML fragment:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
processes:
  -
    host: 127.0.0.1
    ports:
      - 20000
    proxy:
      kind: fault_injection
      host: 127.0.0.1
      port: 12321
      log: proxy_server.log
      wait: 1
      options:
        delay: 5
🖥️ Server Proxies

Add this to an existing server configuration:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.server do |s|
    s.host = '127.0.0.1'
    s.ports = [20_000]

    s.proxy = {
      kind: 'fault_injection',
      host: '127.0.0.1',
      port: 12_321,
      log: 'proxy_server.log',
      wait: 1,
      options: {
        delay: 5
      }
    }
  end
end

YAML fragment:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
servers:
  -
    host: 127.0.0.1
    ports:
      - 20000
    proxy:
      kind: fault_injection
      host: 127.0.0.1
      port: 12321
      log: proxy_server.log
      wait: 1
      options:
        delay: 5
🧩 Service Proxies

Add this to an existing service configuration:

require 'nonnative'

Nonnative.configure do |config|
  config.version = '1.0'
  config.name = 'test'
  config.url = 'http://localhost:4567'
  config.log = 'nonnative.log'

  config.service do |s|
    s.name = 'redis'
    s.host = '127.0.0.1'
    s.ports = [16_379]

    s.proxy = {
      kind: 'fault_injection',
      host: '127.0.0.1',
      port: 6379,
      log: 'proxy_server.log',
      wait: 1,
      options: {
        delay: 5
      }
    }
  end
end

YAML fragment:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
services:
  -
    name: redis
    host: 127.0.0.1
    ports:
      - 16379
    proxy:
      kind: fault_injection
      host: 127.0.0.1
      port: 6379
      log: proxy_server.log
      wait: 1
      options:
        delay: 5
🧪 Fault Injection

The fault_injection proxy allows you to simulate failures by injecting them. We currently support the following:

Clients connect to the runner host and first configured ports entry, while the proxy forwards traffic to nested proxy.host/proxy.port.

  • close_all - Closes the socket as soon as it connects.
  • delay - Delays traffic on the connection. Defaults to 2 seconds and can be configured through options.
  • invalid_data - Forwards client requests unchanged, then corrupts upstream responses before they reach the client.
⚙️ Fault Injection Processes

Set it up programmatically:

name = 'name of process in configuration'
server = Nonnative.pool.process_by_name(name)

server.proxy.close_all # To use close_all.
server.proxy.reset # To reset it back to a good state.

With cucumber:

Given I set the proxy for process 'process_1' to 'close_all'
Then I should reset the proxy for process 'process_1'
🖥️ Fault Injection Servers

Set it up programmatically:

name = 'name of server in configuration'
server = Nonnative.pool.server_by_name(name)

server.proxy.close_all # To use close_all.
server.proxy.reset # To reset it back to a good state.

With cucumber:

Given I set the proxy for server 'server_1' to 'close_all'
Then I should reset the proxy for server 'server_1'
🧩 Fault Injection Services

Set it up programmatically:

name = 'name of service in configuration'
service = Nonnative.pool.service_by_name(name)

service.proxy.close_all # To use close_all.
service.proxy.reset # To reset it back to a good state.

With cucumber:

Given I set the proxy for service 'service_1' to 'close_all'
Then I should reset the proxy for service 'service_1'

🐹 Go

As we love using Go as a language for services we have added support to start binaries with defined parameters.

Programmatic Go binaries can be configured as normal argv process commands:

Nonnative.configure do |config|
  config.process do |p|
    p.name = 'go'
    p.command = -> { Nonnative.go_argv(%w[cover], 'reports', 'your_binary', 'sub_command', '-i file:.config/server.yml') }
    p.ports = [12_345]
  end
end

Use Nonnative.go_argv(...) when a process should execute without shell interpretation, and Nonnative.go_command(...) when a caller needs a command string for Ruby's shell-style spawn behavior.

YAML go: configuration is for Go test binaries compiled with go test -c. It builds argv entries in this order: executable, optional -test.* profiling/trace/coverage flags, command, then parameters. Parameter strings are parsed into argv words with shell-style quoting, but the argv entries are executed without shell interpretation.

[!IMPORTANT] If tools is omitted or empty, Nonnative enables all Go tools: prof, trace, and cover. Provide a subset, such as tools: [cover], to limit the generated -test.* flags.

To get this to work you will need to create a main_test.go file with these contents:

//go:build features
// +build features

package main

import "testing"

func TestFeatures(t *testing.T) {
    main()
}

Then to compile this binary you will need to do the following:

go test -mod vendor -c -tags features -covermode=count -o your_binary -coverpkg=./... github.com/your_location

Set it up through configuration:

version: "1.0"
name: test
url: http://localhost:4567
log: nonnative.log
processes:
  -
    name: go
    go:
      tools: [prof, trace, cover]
      output: reports
      executable: your_binary
      command: sub_command
      parameters:
        - "-i file:.config/server.yml"
    timeout: 5
    ports:
      - 8000
    log: go.log