Class: Vangrail::Client

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/client.rb,
lib/vangrail/client/completion.rb

Overview

Interop with a NeMo Guardrails server that already exists.

Nothing in this gem needs one. It is here for the case where a team already runs the Python service, wants its configs to stay the source of truth, and wants Ruby to call rather than reimplement. Reach for Config#engine first: it runs the same folder in this process with nothing to deploy.

/v1/checks is the endpoint that matches what a rail actually wants, and it answers in the same three states this gem models: passed, modified, blocked. Older servers do not have it, so check falls back to a chat completion with generation switched off and reads the rail-tracking variables out of that.

Defined Under Namespace

Classes: Completion

Constant Summary collapse

CONFIGS_PATH = 
'/v1/rails/configs'
CHECKS_PATH = 
'/v1/checks'
COMPLETIONS_PATH = 
'/v1/chat/completions'
PROTOCOLS = 
%i[auto nested flat].freeze
RAIL_VARS = 
[Completion::INPUT_RAIL_VAR, Completion::OUTPUT_RAIL_VAR].freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(base_url:, config_id: nil, model: nil, api_key: nil, protocol: :auto, open_timeout: HTTP::DEFAULT_OPEN_TIMEOUT, read_timeout: HTTP::DEFAULT_READ_TIMEOUT, http: nil) ⇒ Client

Returns a new instance of Client.



34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/vangrail/client.rb', line 34

def initialize(base_url:, config_id: nil, model: nil, api_key: nil, protocol: :auto,
               open_timeout: HTTP::DEFAULT_OPEN_TIMEOUT, read_timeout: HTTP::DEFAULT_READ_TIMEOUT,
               http: nil)
  unless PROTOCOLS.include?(protocol)
    raise ArgumentError,
          "protocol must be one of #{PROTOCOLS.join(', ')}"
  end

  @config_id = config_id
  @model = model
  @protocol = protocol
  @checks_supported = nil
  @http = http || HTTP.new(
    base_url: base_url, api_key: api_key,
    open_timeout: open_timeout, read_timeout: read_timeout
  )
end

Instance Attribute Details

#checks_supported ⇒ Object (readonly)

True once /v1/checks has answered, false once it has 404ed, nil until one of those happens, so a caller can report "not yet known" honestly.



32
33
34
 
# File 'lib/vangrail/client.rb', line 32

def checks_supported
  @checks_supported
end

#config_id ⇒ Object (readonly)

Returns the value of attribute config_id.



28
29
30
 
# File 'lib/vangrail/client.rb', line 28

def config_id
  @config_id
end

#http ⇒ Object (readonly)

Returns the value of attribute http.



28
29
30
 
# File 'lib/vangrail/client.rb', line 28

def http
  @http
end

#model ⇒ Object (readonly)

Returns the value of attribute model.



28
29
30
 
# File 'lib/vangrail/client.rb', line 28

def model
  @model
end

#protocol ⇒ Object (readonly)

Returns the value of attribute protocol.



28
29
30
 
# File 'lib/vangrail/client.rb', line 28

def protocol
  @protocol
end

Instance Method Details

#available? ⇒ Boolean

Returns:

  • (Boolean) 


62
63
64
 
# File 'lib/vangrail/client.rb', line 62

def available?
  http.reachable?(CONFIGS_PATH)
end

#base_url ⇒ Object 



52
53
54
 
# File 'lib/vangrail/client.rb', line 52

def base_url
  http.base_url
end

#chat(messages:, config_id: nil, config_ids: nil, options: nil, context: nil, thread_id: nil, model: nil, **extra) ⇒ Object

A full guardrailed completion, for the case where the server generates the answer as well as checking it.



94
95
96
97
98
99
100
 
# File 'lib/vangrail/client.rb', line 94

def chat(messages:, config_id: nil, config_ids: nil, options: nil, context: nil,
         thread_id: nil, model: nil, **extra)
  opts = merge_options(options)
  body = extra.merge(messages: normalize(messages))
  chosen = { config_id: config_id || @config_id, config_ids: config_ids }
  Completion.new(send_payload(body, chosen, opts, context, thread_id, model))
end

#check(messages, rail:, config_id: nil) ⇒ Object

Runs rails without generation and returns a Result.



78
79
80
81
82
83
84
85
86
87
88
89
90
 
# File 'lib/vangrail/client.rb', line 78

def check(messages, rail:, config_id: nil)
  chosen = config_id || @config_id
  if @checks_supported != false
    begin
      return from_checks(http.post_json(CHECKS_PATH, checks_payload(messages, rail, chosen)), rail)
    rescue HTTPError => e
      raise unless e.status == 404

      @checks_supported = false
    end
  end
  from_completion(chat(messages: messages, config_id: chosen, options: check_options(rail)), rail)
end

#check_input(text, config_id: nil) ⇒ Object 



66
67
68
 
# File 'lib/vangrail/client.rb', line 66

def check_input(text, config_id: nil)
  check([{ 'role' => 'user', 'content' => text.to_s }], rail: :input, config_id: config_id)
end

#check_output(text, user_input: nil, config_id: nil) ⇒ Object 



70
71
72
73
74
75
 
# File 'lib/vangrail/client.rb', line 70

def check_output(text, user_input: nil, config_id: nil)
  messages = []
  messages << { 'role' => 'user', 'content' => user_input.to_s } unless user_input.to_s.strip.empty?
  messages << { 'role' => 'assistant', 'content' => text.to_s }
  check(messages, rail: :output, config_id: config_id)
end

#configs ⇒ Object 



56
57
58
59
60
 
# File 'lib/vangrail/client.rb', line 56

def configs
  body = http.get_json(CONFIGS_PATH)
  list = body.is_a?(Array) ? body : Array(body['configs'])
  list.filter_map { |entry| entry.is_a?(Hash) ? entry['id'] : entry.to_s }
end