Class: Vangrail::Client

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/client.rb,
lib/vangrail/client/turn.rb

Overview

Interop with a NeMo Guardrails server that already exists.

Nothing in this gem needs one. It is here for the case where a team already runs the Python service, wants its configs to stay the source of truth, and wants Ruby to call rather than reimplement. Reach for Config#engine first: it runs the same folder in this process with nothing to deploy.

/v1/checks is the endpoint that matches what a rail actually wants, and it answers in the same three states this gem models: passed, modified, blocked. Older servers do not have it, so check falls back to a chat completion with generation switched off and reads the rail-tracking variables out of that.

Defined Under Namespace

Classes: Request, Turn

Constant Summary collapse

CONFIGS_PATH = 
'/v1/rails/configs'
CHECKS_PATH = 
'/v1/checks'
COMPLETIONS_PATH = 
'/v1/chat/completions'
PROTOCOLS = 
%i[auto nested flat].freeze
RAIL_VARS = 
[Turn::INPUT_RAIL_VAR, Turn::OUTPUT_RAIL_VAR].freeze
Completion =

The 0.1.0 name. Prefer Turn in new code.

Turn

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(http: nil, base_url: nil, config_id: nil, model: nil, api_key: nil, protocol: :auto, open_timeout: HTTP::DEFAULT_OPEN_TIMEOUT, read_timeout: HTTP::DEFAULT_READ_TIMEOUT) ⇒ Client

Returns a new instance of Client.



41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/vangrail/client.rb', line 41

def initialize(http: nil, base_url: nil, config_id: nil, model: nil, api_key: nil,
               protocol: :auto, open_timeout: HTTP::DEFAULT_OPEN_TIMEOUT,
               read_timeout: HTTP::DEFAULT_READ_TIMEOUT)
  unless PROTOCOLS.include?(protocol)
    raise ArgumentError,
          "protocol must be one of #{PROTOCOLS.join(', ')}"
  end

  @config_id = config_id
  @model = model
  @protocol = protocol
  @checks_supported = nil
  @http = HTTP.build(http: http, base_url: base_url, api_key: api_key,
                     open_timeout: open_timeout, read_timeout: read_timeout,
                     missing: 'a Client needs a base_url or an http client')
end

Instance Attribute Details

#checks_supported ⇒ Object (readonly)

True once /v1/checks has answered, false once it has 404ed, nil until one of those happens, so a caller can report "not yet known" honestly.



36
37
38
 
# File 'lib/vangrail/client.rb', line 36

def checks_supported
  @checks_supported
end

#config_id ⇒ Object (readonly)

Returns the value of attribute config_id.



32
33
34
 
# File 'lib/vangrail/client.rb', line 32

def config_id
  @config_id
end

#http ⇒ Object (readonly)

Returns the value of attribute http.



32
33
34
 
# File 'lib/vangrail/client.rb', line 32

def http
  @http
end

#model ⇒ Object (readonly)

Returns the value of attribute model.



32
33
34
 
# File 'lib/vangrail/client.rb', line 32

def model
  @model
end

#protocol ⇒ Object (readonly)

Returns the value of attribute protocol.



32
33
34
 
# File 'lib/vangrail/client.rb', line 32

def protocol
  @protocol
end

Instance Method Details

#available? ⇒ Boolean

Returns:

  • (Boolean) 


68
69
70
 
# File 'lib/vangrail/client.rb', line 68

def available?
  http.reachable?(CONFIGS_PATH)
end

#base_url ⇒ Object 



58
59
60
 
# File 'lib/vangrail/client.rb', line 58

def base_url
  http.base_url
end

#chat(messages:, config_id: nil, config_ids: nil, options: nil, context: nil, thread_id: nil, model: nil, **extra) ⇒ Object

A full guardrailed completion, for the case where the server generates the answer as well as checking it.



100
101
102
103
104
105
106
107
108
109
 
# File 'lib/vangrail/client.rb', line 100

def chat(messages:, config_id: nil, config_ids: nil, options: nil, context: nil,
         thread_id: nil, model: nil, **extra)
  request = Request.new(messages: messages, config_id: config_id, config_ids: config_ids,
                        options: options, context: context, thread_id: thread_id,
                        model: model, extra: extra)
  opts = merge_options(request.options)
  body = request.extra.merge(messages: normalize(request.messages))
  chosen = { config_id: request.config_id || @config_id, config_ids: request.config_ids }
  Turn.new(send_payload(body, chosen, opts, request.context, request.thread_id, request.model))
end

#check(messages, rail:, config_id: nil) ⇒ Object

Runs rails without generation and returns a Result.



84
85
86
87
88
89
90
91
92
93
94
95
96
 
# File 'lib/vangrail/client.rb', line 84

def check(messages, rail:, config_id: nil)
  chosen = config_id || @config_id
  if @checks_supported != false
    begin
      return from_checks(http.post_json(CHECKS_PATH, checks_payload(messages, rail, chosen)), rail)
    rescue HTTPError => e
      raise unless e.status == 404

      @checks_supported = false
    end
  end
  from_completion(chat(messages: messages, config_id: chosen, options: check_options(rail)), rail)
end

#check_input(text, config_id: nil) ⇒ Object 



72
73
74
 
# File 'lib/vangrail/client.rb', line 72

def check_input(text, config_id: nil)
  check([{ 'role' => 'user', 'content' => text.to_s }], rail: :input, config_id: config_id)
end

#check_output(text, user_input: nil, config_id: nil) ⇒ Object 



76
77
78
79
80
81
 
# File 'lib/vangrail/client.rb', line 76

def check_output(text, user_input: nil, config_id: nil)
  messages = []
  messages << { 'role' => 'user', 'content' => user_input.to_s } unless user_input.to_s.strip.empty?
  messages << { 'role' => 'assistant', 'content' => text.to_s }
  check(messages, rail: :output, config_id: config_id)
end

#configs ⇒ Object 



62
63
64
65
66
 
# File 'lib/vangrail/client.rb', line 62

def configs
  body = http.get_json(CONFIGS_PATH)
  list = body.is_a?(Array) ? body : Array(body['configs'])
  list.filter_map { |entry| entry.is_a?(Hash) ? entry['id'] : entry.to_s }
end