Class: Vangrail::Provider

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/provider.rb

Overview

Where the model-backed rails call, and what they may ask for there.

Every endpoint this gem talks to is OpenAI-compatible, so the differences that matter are not protocol at all. They are: how a credential resolves, whether the endpoint is up, and which model roles it can actually serve. A local proxy is named by environment and may need starting; a shared gateway resolves a token from three places and is either up or not; neither necessarily hosts a safety classifier.

That last point drives a real decision rather than a label. model(:guard) returning nil means the provider has no classifier, and the builder puts a policy rail on the input side instead of pretending a classifier is there.

provider = Vangrail::Provider.resolve          # from the environment
provider.chat(:judge)                                # => Chat, ready to ask

Constant Summary collapse

ROLES = 
%i[guard judge embed].freeze
ENDPOINTS = 
{ chat: Chat, embeddings: Embeddings, completion: Completion }.freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(name:, base_url:, models: {}, key_resolver: nil, guard_preset: nil, local: false, probe: nil) ⇒ Provider

Returns a new instance of Provider.



107
108
109
110
111
112
113
114
115
116
 
# File 'lib/vangrail/provider.rb', line 107

def initialize(name:, base_url:, models: {}, key_resolver: nil, guard_preset: nil,
               local: false, probe: nil)
  @name = name.to_s
  @base_url = base_url.to_s.sub(/\/+\z/, '')
  @models = models
  @key_resolver = key_resolver
  @guard_preset = guard_preset
  @local = local
  @probe = probe
end

Instance Attribute Details

#base_url ⇒ Object (readonly)

Returns the value of attribute base_url.



105
106
107
 
# File 'lib/vangrail/provider.rb', line 105

def base_url
  @base_url
end

#guard_preset ⇒ Object (readonly)

Returns the value of attribute guard_preset.



105
106
107
 
# File 'lib/vangrail/provider.rb', line 105

def guard_preset
  @guard_preset
end

#local ⇒ Object (readonly)

Returns the value of attribute local.



105
106
107
 
# File 'lib/vangrail/provider.rb', line 105

def local
  @local
end

#models ⇒ Object (readonly)

Returns the value of attribute models.



105
106
107
 
# File 'lib/vangrail/provider.rb', line 105

def models
  @models
end

#name ⇒ Object (readonly)

Returns the value of attribute name.



105
106
107
 
# File 'lib/vangrail/provider.rb', line 105

def name
  @name
end

Class Method Details

.[](name) ⇒ Object 



40
41
42
 
# File 'lib/vangrail/provider.rb', line 40

def self.[](name)
  registry[name.to_s]
end

.from_env_pair(env) ⇒ Object

An endpoint given directly, which is how anything unregistered is used.



87
88
89
90
91
92
93
94
95
96
97
98
 
# File 'lib/vangrail/provider.rb', line 87

def self.from_env_pair(env)
  base = present(env['GUARDRAILS_API_BASE'])
  return nil unless base

  new(
    name: 'env',
    base_url: base,
    key_resolver: -> { present(env['GUARDRAILS_API_KEY']) },
    models: { judge: present(env['GUARDRAILS_JUDGE_MODEL']), guard: present(env['GUARDRAILS_MODEL']),
              embed: present(env['GUARDRAILS_EMBED_MODEL']) },
  )
end

.gateway_in(env) ⇒ Object

A gateway described by the environment this call was handed, rather than by the one the registry happened to be installed from. Resolution is then a function of (registry, env), which is what a caller passing an env hash is entitled to assume.



79
80
81
82
83
84
 
# File 'lib/vangrail/provider.rb', line 79

def self.gateway_in(env)
  return nil if env.equal?(ENV)

  spec = Providers::Gateway.from_environment(env)
  spec && Providers::Gateway.provider(spec, env)
end

.names ⇒ Object 



44
45
46
 
# File 'lib/vangrail/provider.rb', line 44

def self.names
  registry.keys
end

.present(value) ⇒ Object 



100
101
102
103
 
# File 'lib/vangrail/provider.rb', line 100

def self.present(value)
  s = value.to_s.strip
  s.empty? ? nil : s
end

.register(provider) ⇒ Object 



35
36
37
38
 
# File 'lib/vangrail/provider.rb', line 35

def self.register(provider)
  registry[provider.name] = provider
  provider
end

.registry ⇒ Object

Presets by name, in the order resolve tries them.



31
32
33
 
# File 'lib/vangrail/provider.rb', line 31

def self.registry
  @registry ||= {}
end

.resolve(env = ENV) ⇒ Object

Picks a provider from the environment.

GUARDRAILS_PROVIDER=<name>   take this one, and fail loudly if it is
                           unknown rather than falling back
GUARDRAILS_API_BASE + key    an endpoint nobody registered
otherwise                    the first registered provider that is
                           actually available, in registration order

Returning nil is a legitimate answer: no endpoint is reachable, and the caller builds an engine with only the offline rails on it.



58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/vangrail/provider.rb', line 58

def self.resolve(env = ENV)
  candidates = registry.each_value.to_a + [gateway_in(env)].compact

  wanted = present(env['GUARDRAILS_PROVIDER'])
  if wanted
    found = candidates.detect { |p| p.name == wanted }
    raise ConfigError, "unknown provider #{wanted.inspect}; known: #{names.join(', ')}" unless found

    return found.with_env(env)
  end

  explicit = from_env_pair(env)
  return explicit if explicit

  candidates.map { |p| p.with_env(env) }.detect(&:available?)
end

Instance Method Details

#api_key ⇒ Object 



137
138
139
140
141
 
# File 'lib/vangrail/provider.rb', line 137

def api_key
  return @api_key if defined?(@api_key)

  @api_key = @key_resolver&.call
end

#available? ⇒ Boolean

Up, and holding a credential. A local endpoint is probed, because a proxy that is not running is the ordinary case rather than a failure.

Returns:

  • (Boolean) 


165
166
167
168
169
170
 
# File 'lib/vangrail/provider.rb', line 165

def available?
  return false unless api_key || !credential_required?
  return true unless @probe

  @probe.call
end

#chat(role = :judge, **kwargs) ⇒ Object 



180
181
182
 
# File 'lib/vangrail/provider.rb', line 180

def chat(role = :judge, **kwargs)
  endpoint(:chat, role, **kwargs)
end

#completion(role = :judge, **kwargs) ⇒ Object

Scoring rather than generation, from whichever model answers questions. No separate role: any causal model can score text, and asking a deployment to name a second one for it would be ceremony.



191
192
193
 
# File 'lib/vangrail/provider.rb', line 191

def completion(role = :judge, **kwargs)
  endpoint(:completion, role, **kwargs)
end

#credential_required? ⇒ Boolean

Returns:

  • (Boolean) 


172
173
174
 
# File 'lib/vangrail/provider.rb', line 172

def credential_required?
  !@key_resolver.nil?
end

#embed? ⇒ Boolean

Can it embed. Named rather than assumed for the same reason guard? is: an endpoint serving chat need not serve embeddings, and a rail built on the assumption that it does is a rail that reports an error instead of a verdict. No default model is guessed either, because the name of an embedding model is deployment knowledge and a wrong guess is a 404 per check.

Returns:

  • (Boolean) 


159
160
161
 
# File 'lib/vangrail/provider.rb', line 159

def embed?
  !model(:embed).nil?
end

#embeddings(role = :embed, **kwargs) ⇒ Object 



184
185
186
 
# File 'lib/vangrail/provider.rb', line 184

def embeddings(role = :embed, **kwargs)
  endpoint(:embeddings, role, **kwargs)
end

#guard? ⇒ Boolean

Can this provider serve a safety classifier, as opposed to an instruct model answering a written policy.

Returns:

  • (Boolean) 


149
150
151
 
# File 'lib/vangrail/provider.rb', line 149

def guard?
  !model(:guard).nil? && !guard_preset.nil?
end

#http ⇒ Object 



176
177
178
 
# File 'lib/vangrail/provider.rb', line 176

def http
  @http ||= HTTP.new(base_url: base_url, api_key: api_key)
end

#model(role) ⇒ Object 



143
144
145
 
# File 'lib/vangrail/provider.rb', line 143

def model(role)
  models[role.to_sym]
end

#to_h ⇒ Object 



195
196
197
198
199
200
201
202
203
 
# File 'lib/vangrail/provider.rb', line 195

def to_h
  {
    'name' => name,
    'base_url' => base_url,
    'models' => models.transform_keys(&:to_s).compact,
    'guard_preset' => guard_preset&.to_s,
    'local' => local,
  }.compact
end

#to_s ⇒ Object 



205
206
207
 
# File 'lib/vangrail/provider.rb', line 205

def to_s
  "#{name} #{base_url}"
end

#with_env(env) ⇒ Object

A copy that reads overrides out of an environment. Providers are shared objects in a registry, so nothing mutates in place.



120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/vangrail/provider.rb', line 120

def with_env(env)
  overrides = {
    judge: self.class.present(env['GUARDRAILS_JUDGE_MODEL']),
    guard: self.class.present(env['GUARDRAILS_MODEL']),
    embed: self.class.present(env['GUARDRAILS_EMBED_MODEL']),
  }.compact
  base = self.class.present(env["#{env_prefix}_API_BASE"]) || base_url
  key = self.class.present(env["#{env_prefix}_API_KEY"])
  return self if overrides.empty? && base == base_url && key.nil?

  self.class.new(
    name: name, base_url: base, models: models.merge(overrides),
    key_resolver: key ? -> { key } : @key_resolver,
    guard_preset: guard_preset, local: local, probe: @probe
  )
end