Class: Labimotion::AiTemplate

Inherits:
Object
  • Object
show all
Defined in:
lib/labimotion/libs/ai_template.rb

Overview

AiTemplate generates a generic dataset template (DatasetKlass properties template) from a natural-language description using an LLM.

The client targets the KIT KI-Toolbox (https://ki-toolbox.scc.kit.edu), an OpenAI-compatible chat-completions gateway (Open WebUI), but is fully config-driven (config/labimotion_ai.yml, then ENV) so it can point at any compatible service. Settings resolve yml -> ENV -> default:

:api_key  / KI_TOOLBOX_API_KEY    (required) bearer token (created in the UI)
:base_url / KI_TOOLBOX_BASE_URL   default https://ki-toolbox.scc.kit.edu
:api_path / KI_TOOLBOX_API_PATH   default /api/v1/chat/completions
:model    / KI_TOOLBOX_MODEL      default azure.gpt-4.1-mini
:max_tokens, :timeout, :system_prompt

The LLM is asked to return a JSON document describing the template layers and fields in the LabIMotion format. Only the data part is returned here; wrapping it into a persistable DatasetKlass (uuid, pkg, klass, ...) is the caller's job (see Labimotion::DatasetHelpers#create_ai_dataset_klass).

Constant Summary collapse

DEFAULT_BASE_URL = 
'https://ki-toolbox.scc.kit.edu'
DEFAULT_API_PATH = 
'/api/v1/chat/completions'
DEFAULT_MODEL = 
'azure.gpt-4.1-mini'
DEFAULT_MAX_TOKENS = 
8000
PING_MAX_TOKENS =

A connection test only needs a valid round-trip, not a useful completion, so cap the reply hard to keep the ping cheap and fast.

8
REQUEST_TIMEOUT = 
120
MAX_FILES = 
10
MAX_FILE_CHARS =

Slightly above FileExtractor::MAX_OUT_CHARS (8000) so its "[truncated]" marker is not chopped off by the slice below.

9000
FIELD_TYPES =

Field "type" values allowed in generated templates — a practical subset of the LabIMotion field-base schema enum.

%w[
  text textarea integer number checkbox date datetime
  select select-multi ontology-select system-defined label upload table
].freeze
SELECT_TYPES =

Field types whose choices live in the shared select_options map.

%w[select select-multi].freeze
UNIT_CANDIDATE_TYPES =

Field types that may carry a physical unit (mapped to a system-defined group).

%w[number integer system-defined].freeze
TABLE_COL_TYPES =

Column types allowed inside a "table" field's sub_fields; others -> text.

%w[text number integer checkbox date datetime].freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], history: [], model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ AiTemplate

Returns a new instance of AiTemplate.



183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
 
# File 'lib/labimotion/libs/ai_template.rb', line 183

def initialize(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], history: [], model: nil, api_key: nil, base_url: nil, api_path: nil)
  @kind = kind.to_s
  @subject = subject.to_s.strip
  @ols_term_id = ols_term_id.to_s.strip
  @desc = desc.to_s.strip
  @cols = cols.to_i.clamp(1, 6)
  @references = Array(references).map(&:to_s).reject(&:blank?)
  @files = Array(files)
  @history = Array(history)
  # Per-user overrides (from the caller); blank -> fall back to yml/ENV/default.
  @model_override = model.to_s.strip
  @api_key_override = api_key.to_s.strip
  # A user may only redirect to their OWN provider when they also bring their
  # own key — the shared server key must never be sent to a user-supplied URL.
  @base_url_override = base_url.to_s.strip
  @api_path_override = api_path.to_s.strip
end

Class Method Details

.build_unit_index ⇒ Object 



141
142
143
144
145
146
147
148
 
# File 'lib/labimotion/libs/ai_template.rb', line 141

def self.build_unit_index
  collect_unit_tokens.each_with_object({}) do |(token, groups), index|
    next unless groups.size == 1

    gkey, ukey = groups.first
    index[token] = { group: gkey, value_system: ukey }
  end
end

.collect_unit_tokens ⇒ Object

token -> { group => first_unit_key }. A token seen under >1 group signals it is ambiguous (kept here, filtered out in build_unit_index).



152
153
154
155
156
157
158
159
 
# File 'lib/labimotion/libs/ai_template.rb', line 152

def self.collect_unit_tokens
  tokens = {}
  Labimotion::Units::FIELDS.each do |group|
    gkey = group[:field].to_s
    Array(group[:units]).each { |unit| record_unit_tokens(tokens, gkey, unit) }
  end
  tokens
end

.fill(properties:, context_text:, model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Extract DATA VALUES for an existing generic element/segment/dataset instance from a document's text, to pre-fill the working copy for human review. This does NOT design or modify a template — it only reads values for the fields the given properties template already defines. Nothing is persisted here.

Parameters:

  • properties (Hash) —

    the element instance properties ({ 'layers' => ..., 'select_options' => ... })

  • context_text (String) —

    plain text extracted from the source document(s)

  • model (String, nil) (defaults to: nil) —

    per-user model override (falls back to yml/ENV/default)

  • api_key (String, nil) (defaults to: nil) —

    per-user API key override (falls back to yml/ENV)

  • base_url (String, nil) (defaults to: nil) —

    per-user provider base URL (honored only with api_key)

  • api_path (String, nil) (defaults to: nil) —

    per-user provider chat-completions path (honored only with api_key)

Returns:

  • (Hash) —

    { 'values' => { layer_key => { field_key => value } }, 'summary' => String }



111
112
113
114
 
# File 'lib/labimotion/libs/ai_template.rb', line 111

def self.fill(properties:, context_text:, model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(ols_term_id: '', model: model, api_key: api_key, base_url: base_url, api_path: api_path)
    .fill(properties: properties, context_text: context_text)
end

.generate(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Generate a metadata template for a generic dataset, element or segment.

The JSON template schema (layers / fields / select_options) is IDENTICAL across all three kinds — only the prompt wording changes. The dataset path is driven by a CHMO ontology term; the element/segment paths are driven by a subject string (the element/segment name/label).

Parameters:

  • kind (String) (defaults to: 'dataset') —

    'dataset' (default), 'element' or 'segment'

  • ols_term_id (String) (defaults to: nil) —

    CHMO term id, e.g. "CHMO:0000470 | mass spectrometry (MS)" (dataset only)

  • subject (String) (defaults to: nil) —

    the element/segment name/label (element/segment only)

  • desc (String) (defaults to: nil) —

    free-text description of the template

  • cols (Integer) (defaults to: 1) —

    default columns per row for generated layers (1-6)

  • references (Array<String>) (defaults to: []) —

    reference links (publications / standards)

  • files (Array<Hash>) (defaults to: []) —

    [{ 'filename' => ..., 'content' => ... }, ...]

Returns:

  • (Hash) —

    { 'label' => String, 'layers' => Hash, 'select_options' => Hash }



71
72
73
74
 
# File 'lib/labimotion/libs/ai_template.rb', line 71

def self.generate(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(kind: kind, ols_term_id: ols_term_id, subject: subject, desc: desc, cols: cols, references: references,
      files: files, model: model, api_key: api_key, base_url: base_url, api_path: api_path).generate
end

.normalize_unit_token(str) ⇒ Object

Fold a unit string to a comparable token: strip HTML (2 -> 2), map unicode superscripts to digits, micro sign to "u", downcase, drop spaces. So "°C", "µmol/L", "g/cm3" match the model's "°c"/"umol/l"/"g/cm3".



176
177
178
179
180
181
 
# File 'lib/labimotion/libs/ai_template.rb', line 176

def self.normalize_unit_token(str)
  s = str.to_s.gsub(/<[^>]+>/, '')
  s = s.tr('²³¹⁰⁴⁵⁶⁷⁸⁹', '2310456789')
  s = s.gsub(/[µμ]/, 'u').downcase
  s.gsub(/\s+/, '')
end

.ping(model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Verify the (per-user or server) AI settings by sending one tiny chat request and reporting whether it round-trips. Nothing is generated or persisted. Same resolution as the real calls: base_url/api_path are honored only ALONGSIDE a personal key (custom_endpoint?), and a custom base_url is SSRF-validated.

Returns:

  • (Hash) —

    { 'ok' => true, 'model' => .., 'endpoint' => .., 'ms' => .. }

Raises:

  • (RuntimeError) —

    with a user-facing message on any failure



123
124
125
 
# File 'lib/labimotion/libs/ai_template.rb', line 123

def self.ping(model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(ols_term_id: '', model: model, api_key: api_key, base_url: base_url, api_path: api_path).ping
end

.record_unit_tokens(tokens, gkey, unit) ⇒ Object 



161
162
163
164
165
166
167
168
169
170
171
 
# File 'lib/labimotion/libs/ai_template.rb', line 161

def self.record_unit_tokens(tokens, gkey, unit)
  ukey = unit[:key].to_s
  return if ukey.empty?

  [unit[:label], ukey].each do |raw|
    token = normalize_unit_token(raw)
    next if token.empty?

    (tokens[token] ||= {})[gkey] ||= ukey
  end
end

.refine(current:, instruction:, ols_term_id: nil, history: [], cols: 1, model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Refine an existing dataset template through a natural-language instruction.

The caller passes the CURRENT template (label, layers, select_options) — which already reflects any previously-applied refinements — plus the latest instruction and an optional short history of prior turns for continuity. The LLM returns the FULL revised template in the same shape, together with a one-line summary of what it changed. Nothing is persisted here; the admin reviews the result in the designer and saves it (human-in-the-loop).

Parameters:

  • current (Hash) —

    { 'label' => String, 'layers' => Hash, 'select_options' => Hash }

  • instruction (String) —

    the change to apply

  • history (Array<Hash>) (defaults to: []) —

    prior turns [{ 'role' => 'user'|'assistant', 'content' => String }]

  • cols (Integer) (defaults to: 1) —

    fallback columns per row when a layer omits its own (1-6)

  • model (String, nil) (defaults to: nil) —

    per-user model override (falls back to yml/ENV/default)

  • api_key (String, nil) (defaults to: nil) —

    per-user API key override (falls back to yml/ENV)

  • base_url (String, nil) (defaults to: nil) —

    per-user provider base URL (honored only with api_key)

  • api_path (String, nil) (defaults to: nil) —

    per-user provider chat-completions path (honored only with api_key)

Returns:

  • (Hash) —

    { 'label' => String, 'layers' => Hash, 'select_options' => Hash, 'summary' => String }



94
95
96
97
 
# File 'lib/labimotion/libs/ai_template.rb', line 94

def self.refine(current:, instruction:, ols_term_id: nil, history: [], cols: 1, model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(ols_term_id: ols_term_id, cols: cols, history: history, model: model, api_key: api_key, base_url: base_url, api_path: api_path)
    .refine(current: current, instruction: instruction)
end

.unit_groups ⇒ Object

All valid unit-group keys (the option_layers value of a system-defined field).



137
138
139
 
# File 'lib/labimotion/libs/ai_template.rb', line 137

def self.unit_groups
  @unit_groups ||= Labimotion::Units::FIELDS.map { |g| g[:field].to_s }.freeze
end

.unit_index ⇒ Object

Reverse index of physical units -> the LabIMotion system-defined unit group, built once from Labimotion::Units::FIELDS. Maps a normalized unit token (from a unit's label OR its key) to { group:, value_system: }. Tokens used by more than one group are AMBIGUOUS and dropped, so a lookup never guesses the wrong group (e.g. "Pa" -> pressure vs elastic modulus falls back to a plain number).



132
133
134
 
# File 'lib/labimotion/libs/ai_template.rb', line 132

def self.unit_index
  @unit_index ||= build_unit_index
end

Instance Method Details

#fill(properties:, context_text:) ⇒ Object 



256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
 
# File 'lib/labimotion/libs/ai_template.rb', line 256

def fill(properties:, context_text:)
  raise 'AI API key is not configured (set config/labimotion_ai.yml :api_key or KI_TOOLBOX_API_KEY)' if api_key.blank?
  raise 'No readable text could be extracted from the selected source' if context_text.to_s.strip.blank?

  props = properties.is_a?(Hash) ? properties : {}
  schema = fill_field_schema(props)
  raise 'This element template has no fields to fill' if schema.empty?

  response = post_chat(fill_messages(schema, context_text))
  raise "AI request failed (HTTP #{response.code})" unless response.code == 200

  body = JSON.parse(response.body)
  guard_finish_reason!(body)

  text = extract_text(body)
  raise 'AI returned an empty response' if text.blank?

  data = parse_json(text)
  raw_values = data.is_a?(Hash) ? data['values'] : nil
  {
    'values' => normalize_fill_values(props, raw_values),
    'summary' => (data.is_a?(Hash) ? data['summary'].to_s.strip : '')
  }
rescue JSON::ParserError => e
  log_ai_response('fill could not parse response', response&.body)
  Labimotion.log_exception(e)
  raise 'AI returned a response that could not be parsed'
end

#generate ⇒ Object 



201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
 
# File 'lib/labimotion/libs/ai_template.rb', line 201

def generate
  raise 'AI API key is not configured (set config/labimotion_ai.yml :api_key or KI_TOOLBOX_API_KEY)' if api_key.blank?
  if @kind == 'dataset'
    raise 'An ontology term (CHMO) is required' if @ols_term_id.blank?
  else
    raise 'A name/label is required' if @subject.blank?
  end

  response = post_messages
  raise "AI request failed (HTTP #{response.code})" unless response.code == 200

  body = JSON.parse(response.body)
  guard_finish_reason!(body)

  text = extract_text(body)
  raise 'AI returned an empty response' if text.blank?

  normalize(parse_json(text))
rescue JSON::ParserError => e
  log_ai_response('generate could not parse response', response&.body)
  Labimotion.log_exception(e)
  raise 'AI returned a response that could not be parsed as a template'
end

#ping ⇒ Object 



285
286
287
288
289
290
291
292
293
294
 
# File 'lib/labimotion/libs/ai_template.rb', line 285

def ping
  raise 'AI API key is not configured' if api_key.blank?

  started  = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  response = post_chat([{ role: 'user', content: 'ping' }], PING_MAX_TOKENS)
  ms       = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).round
  raise ping_error(response.code) unless response.code == 200

  { 'ok' => true, 'model' => model, 'endpoint' => "#{base_url}#{api_path}", 'ms' => ms }
end

#refine(current:, instruction:) ⇒ Object 



225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
 
# File 'lib/labimotion/libs/ai_template.rb', line 225

def refine(current:, instruction:)
  raise 'AI API key is not configured (set config/labimotion_ai.yml :api_key or KI_TOOLBOX_API_KEY)' if api_key.blank?
  raise 'An instruction is required' if instruction.to_s.strip.blank?

  response = post_chat(refine_messages(current, instruction))
  raise "AI request failed (HTTP #{response.code})" unless response.code == 200

  body = JSON.parse(response.body)
  guard_finish_reason!(body)

  text = extract_text(body)
  raise 'AI returned an empty response' if text.blank?

  data = parse_json(text)
  result = normalize_preserving(data)
  # Guard against a malformed response silently wiping the template — a
  # dataset template always has at least one layer.
  if result['layers'].empty?
    log_ai_response('refine produced no layers', text)
    raise 'AI returned an empty template (the model response contained no layers). ' \
          'Try a more capable model.'
  end

  result['summary'] = (data.is_a?(Hash) ? data['summary'].to_s.strip : '')
  result
rescue JSON::ParserError => e
  log_ai_response('refine could not parse response', response&.body)
  Labimotion.log_exception(e)
  raise 'AI returned a response that could not be parsed as a template'
end