Class: Labimotion::AiTemplate

Inherits:
Object
  • Object
show all
Defined in:
lib/labimotion/libs/ai_template.rb

Overview

AiTemplate generates a generic dataset template (DatasetKlass properties template) from a natural-language description using an LLM.

The client targets the KIT KI-Toolbox (https://ki-toolbox.scc.kit.edu), an OpenAI-compatible chat-completions gateway (Open WebUI), but is fully config-driven (config/labimotion_ai.yml, then ENV) so it can point at any compatible service. Settings resolve yml -> ENV -> default:

:api_key  / KI_TOOLBOX_API_KEY    (required) bearer token (created in the UI)
:base_url / KI_TOOLBOX_BASE_URL   default https://ki-toolbox.scc.kit.edu
:api_path / KI_TOOLBOX_API_PATH   default /api/v1/chat/completions
:model    / KI_TOOLBOX_MODEL      default azure.gpt-4.1-mini
:max_tokens, :timeout, :system_prompt

The LLM is asked to return a JSON document describing the template layers and fields in the LabIMotion format. Only the data part is returned here; wrapping it into a persistable DatasetKlass (uuid, pkg, klass, ...) is the caller's job (see Labimotion::DatasetHelpers#create_ai_dataset_klass).

Constant Summary collapse

DEFAULT_BASE_URL = 
'https://ki-toolbox.scc.kit.edu'
DEFAULT_API_PATH = 
'/api/v1/chat/completions'
DEFAULT_MODEL = 
'azure.gpt-4.1-mini'
DEFAULT_MAX_TOKENS = 
8000
PING_MAX_TOKENS =

A connection test only needs a valid round-trip, not a useful completion, so cap the reply hard to keep the ping cheap and fast.

8
REQUEST_TIMEOUT = 
120
MAX_FILES = 
10
MAX_FILE_CHARS =

Slightly above FileExtractor::MAX_OUT_CHARS (8000) so its "[truncated]" marker is not chopped off by the slice below.

9000
MAX_INSTRUCTION_CHARS =

Cap on a template's own auto-fill guidance. Generous for the paragraph or two this is meant to hold, small enough that it cannot crowd out the document.

4000
MAX_ONTOLOGY_TERMS =

Ontology vocabularies are sent verbatim; past this many terms in ONE request the remaining ontology fields go unconstrained rather than starve the document of context. Sized well above a realistic template (Viability: ~470 terms).

Labimotion::OntologyTerms::MAX_TERMS_TOTAL
FIELD_TYPES =

Field "type" values allowed in generated templates — a practical subset of the LabIMotion field-base schema enum.

%w[
  text textarea integer number checkbox date datetime
  select select-multi ontology-select system-defined label upload table
].freeze
SELECT_TYPES =

Field types whose choices live in the shared select_options map.

%w[select select-multi].freeze
UNIT_CANDIDATE_TYPES =

Field types that may carry a physical unit (mapped to a system-defined group).

%w[number integer system-defined].freeze
TABLE_COL_TYPES =

Column types allowed inside a "table" field's sub_fields; others -> text.

%w[text number integer checkbox date datetime].freeze
DURATION_UNITS =

The duration units a "datetime-range" field can carry, as chem-generic-ui's DateTimeRange renders them. The model picks one alongside its number, so "three days" cannot land in a field that then reads it as three hours.

%w[d h min s].freeze
MAX_TABLE_ROWS =

Rows past this are dropped: a table is a summary of a document, and a model that starts emitting hundreds of rows has lost the thread.

50
MAX_FIELD_DESC_CHARS =

Per-field cap on the designer's description. Generous enough to carry a full ontology definition intact (the longest in the Viability template is 771), while still bounding a template that puts an essay on every field.

1000

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], history: [], model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ AiTemplate

Returns a new instance of AiTemplate.



205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
 
# File 'lib/labimotion/libs/ai_template.rb', line 205

def initialize(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], history: [], model: nil, api_key: nil, base_url: nil, api_path: nil)
  @kind = kind.to_s
  @subject = subject.to_s.strip
  @ols_term_id = ols_term_id.to_s.strip
  @desc = desc.to_s.strip
  @cols = cols.to_i.clamp(1, 6)
  @references = Array(references).map(&:to_s).reject(&:blank?)
  @files = Array(files)
  @history = Array(history)
  # Per-user overrides (from the caller); blank -> fall back to yml/ENV/default.
  @model_override = model.to_s.strip
  @api_key_override = api_key.to_s.strip
  # A user may only redirect to their OWN provider when they also bring their
  # own key — the shared server key must never be sent to a user-supplied URL.
  @base_url_override = base_url.to_s.strip
  @api_path_override = api_path.to_s.strip
end

Class Method Details

.build_unit_index ⇒ Object 



163
164
165
166
167
168
169
170
 
# File 'lib/labimotion/libs/ai_template.rb', line 163

def self.build_unit_index
  collect_unit_tokens.each_with_object({}) do |(token, groups), index|
    next unless groups.size == 1

    gkey, ukey = groups.first
    index[token] = { group: gkey, value_system: ukey }
  end
end

.collect_unit_tokens ⇒ Object

token -> { group => first_unit_key }. A token seen under >1 group signals it is ambiguous (kept here, filtered out in build_unit_index).



174
175
176
177
178
179
180
181
 
# File 'lib/labimotion/libs/ai_template.rb', line 174

def self.collect_unit_tokens
  tokens = {}
  Labimotion::Units::FIELDS.each do |group|
    gkey = group[:field].to_s
    Array(group[:units]).each { |unit| record_unit_tokens(tokens, gkey, unit) }
  end
  tokens
end

.fill(properties:, context_text:, instructions: nil, model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Extract DATA VALUES for an existing generic element/segment/dataset instance from a document's text, to pre-fill the working copy for human review. This does NOT design or modify a template — it only reads values for the fields the given properties template already defines. Nothing is persisted here.

Parameters:

  • properties (Hash) —

    the element instance properties ({ 'layers' => ..., 'select_options' => ... })

  • context_text (String) —

    plain text extracted from the source document(s)

  • instructions (String, nil) (defaults to: nil) —

    the TEMPLATE's own extraction guidance, set by the designer in Template settings (klass.settings). Appended to the system prompt; it steers WHAT to look for, never the response format.

  • model (String, nil) (defaults to: nil) —

    per-user model override (falls back to yml/ENV/default)

  • api_key (String, nil) (defaults to: nil) —

    per-user API key override (falls back to yml/ENV)

  • base_url (String, nil) (defaults to: nil) —

    per-user provider base URL (honored only with api_key)

  • api_path (String, nil) (defaults to: nil) —

    per-user provider chat-completions path (honored only with api_key)

Returns:

  • (Hash) —

    { 'values' => { layer_key => { field_key => value } }, 'summary' => String }



133
134
135
136
 
# File 'lib/labimotion/libs/ai_template.rb', line 133

def self.fill(properties:, context_text:, instructions: nil, model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(ols_term_id: '', model: model, api_key: api_key, base_url: base_url, api_path: api_path)
    .fill(properties: properties, context_text: context_text, instructions: instructions)
end

.generate(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Generate a metadata template for a generic dataset, element or segment.

The JSON template schema (layers / fields / select_options) is IDENTICAL across all three kinds — only the prompt wording changes. The dataset path is driven by a CHMO ontology term; the element/segment paths are driven by a subject string (the element/segment name/label).

Parameters:

  • kind (String) (defaults to: 'dataset') —

    'dataset' (default), 'element' or 'segment'

  • ols_term_id (String) (defaults to: nil) —

    CHMO term id, e.g. "CHMO:0000470 | mass spectrometry (MS)" (dataset only)

  • subject (String) (defaults to: nil) —

    the element/segment name/label (element/segment only)

  • desc (String) (defaults to: nil) —

    free-text description of the template

  • cols (Integer) (defaults to: 1) —

    default columns per row for generated layers (1-6)

  • references (Array<String>) (defaults to: []) —

    reference links (publications / standards)

  • files (Array<Hash>) (defaults to: []) —

    [{ 'filename' => ..., 'content' => ... }, ...]

Returns:

  • (Hash) —

    { 'label' => String, 'layers' => Hash, 'select_options' => Hash }



90
91
92
93
 
# File 'lib/labimotion/libs/ai_template.rb', line 90

def self.generate(kind: 'dataset', ols_term_id: nil, subject: nil, desc: nil, cols: 1, references: [], files: [], model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(kind: kind, ols_term_id: ols_term_id, subject: subject, desc: desc, cols: cols, references: references,
      files: files, model: model, api_key: api_key, base_url: base_url, api_path: api_path).generate
end

.normalize_unit_token(str) ⇒ Object

Fold a unit string to a comparable token: strip HTML (2 -> 2), map unicode superscripts to digits, micro sign to "u", downcase, drop spaces. So "°C", "µmol/L", "g/cm3" match the model's "°c"/"umol/l"/"g/cm3".



198
199
200
201
202
203
 
# File 'lib/labimotion/libs/ai_template.rb', line 198

def self.normalize_unit_token(str)
  s = str.to_s.gsub(/<[^>]+>/, '')
  s = s.tr('²³¹⁰⁴⁵⁶⁷⁸⁹', '2310456789')
  s = s.gsub(/[µμ]/, 'u').downcase
  s.gsub(/\s+/, '')
end

.ping(model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Verify the (per-user or server) AI settings by sending one tiny chat request and reporting whether it round-trips. Nothing is generated or persisted. Same resolution as the real calls: base_url/api_path are honored only ALONGSIDE a personal key (custom_endpoint?), and a custom base_url is SSRF-validated.

Returns:

  • (Hash) —

    { 'ok' => true, 'model' => .., 'endpoint' => .., 'ms' => .. }

Raises:

  • (RuntimeError) —

    with a user-facing message on any failure



145
146
147
 
# File 'lib/labimotion/libs/ai_template.rb', line 145

def self.ping(model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(ols_term_id: '', model: model, api_key: api_key, base_url: base_url, api_path: api_path).ping
end

.record_unit_tokens(tokens, gkey, unit) ⇒ Object 



183
184
185
186
187
188
189
190
191
192
193
 
# File 'lib/labimotion/libs/ai_template.rb', line 183

def self.record_unit_tokens(tokens, gkey, unit)
  ukey = unit[:key].to_s
  return if ukey.empty?

  [unit[:label], ukey].each do |raw|
    token = normalize_unit_token(raw)
    next if token.empty?

    (tokens[token] ||= {})[gkey] ||= ukey
  end
end

.refine(current:, instruction:, ols_term_id: nil, history: [], cols: 1, model: nil, api_key: nil, base_url: nil, api_path: nil) ⇒ Hash

Refine an existing dataset template through a natural-language instruction.

The caller passes the CURRENT template (label, layers, select_options) — which already reflects any previously-applied refinements — plus the latest instruction and an optional short history of prior turns for continuity. The LLM returns the FULL revised template in the same shape, together with a one-line summary of what it changed. Nothing is persisted here; the admin reviews the result in the designer and saves it (human-in-the-loop).

Parameters:

  • current (Hash) —

    { 'label' => String, 'layers' => Hash, 'select_options' => Hash }

  • instruction (String) —

    the change to apply

  • history (Array<Hash>) (defaults to: []) —

    prior turns [{ 'role' => 'user'|'assistant', 'content' => String }]

  • cols (Integer) (defaults to: 1) —

    fallback columns per row when a layer omits its own (1-6)

  • model (String, nil) (defaults to: nil) —

    per-user model override (falls back to yml/ENV/default)

  • api_key (String, nil) (defaults to: nil) —

    per-user API key override (falls back to yml/ENV)

  • base_url (String, nil) (defaults to: nil) —

    per-user provider base URL (honored only with api_key)

  • api_path (String, nil) (defaults to: nil) —

    per-user provider chat-completions path (honored only with api_key)

Returns:

  • (Hash) —

    { 'label' => String, 'layers' => Hash, 'select_options' => Hash, 'summary' => String }



113
114
115
116
 
# File 'lib/labimotion/libs/ai_template.rb', line 113

def self.refine(current:, instruction:, ols_term_id: nil, history: [], cols: 1, model: nil, api_key: nil, base_url: nil, api_path: nil)
  new(ols_term_id: ols_term_id, cols: cols, history: history, model: model, api_key: api_key, base_url: base_url, api_path: api_path)
    .refine(current: current, instruction: instruction)
end

.unit_groups ⇒ Object

All valid unit-group keys (the option_layers value of a system-defined field).



159
160
161
 
# File 'lib/labimotion/libs/ai_template.rb', line 159

def self.unit_groups
  @unit_groups ||= Labimotion::Units::FIELDS.map { |g| g[:field].to_s }.freeze
end

.unit_index ⇒ Object

Reverse index of physical units -> the LabIMotion system-defined unit group, built once from Labimotion::Units::FIELDS. Maps a normalized unit token (from a unit's label OR its key) to { group:, value_system: }. Tokens used by more than one group are AMBIGUOUS and dropped, so a lookup never guesses the wrong group (e.g. "Pa" -> pressure vs elastic modulus falls back to a plain number).



154
155
156
 
# File 'lib/labimotion/libs/ai_template.rb', line 154

def self.unit_index
  @unit_index ||= build_unit_index
end

Instance Method Details

#fill(properties:, context_text:, instructions: nil) ⇒ Object 



278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
 
# File 'lib/labimotion/libs/ai_template.rb', line 278

def fill(properties:, context_text:, instructions: nil)
  raise 'AI API key is not configured (set config/labimotion_ai.yml :api_key or KI_TOOLBOX_API_KEY)' if api_key.blank?
  raise 'No readable text could be extracted from the selected source' if context_text.to_s.strip.blank?

  props = properties.is_a?(Hash) ? properties : {}
  @fill_instructions = instructions.to_s.strip
  # Resolved ONCE and reused by both the prompt and the validation below, so the
  # model is never offered a term the answer is then rejected against.
  @ontology_vocabulary = build_ontology_vocabulary(props)
  schema = fill_field_schema(props)
  raise 'This element template has no fields to fill' if schema.empty?

  response = post_chat(fill_messages(schema, context_text))
  raise "AI request failed (HTTP #{response.code})" unless response.code == 200

  body = JSON.parse(response.body)
  guard_finish_reason!(body)

  text = extract_text(body)
  raise 'AI returned an empty response' if text.blank?

  data = parse_json(text)
  raw_values = data.is_a?(Hash) ? data['values'] : nil
  {
    'values' => normalize_fill_values(props, raw_values),
    'summary' => (data.is_a?(Hash) ? data['summary'].to_s.strip : '')
  }
rescue JSON::ParserError => e
  log_ai_response('fill could not parse response', response&.body)
  Labimotion.log_exception(e)
  raise 'AI returned a response that could not be parsed'
end

#generate ⇒ Object 



223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
 
# File 'lib/labimotion/libs/ai_template.rb', line 223

def generate
  raise 'AI API key is not configured (set config/labimotion_ai.yml :api_key or KI_TOOLBOX_API_KEY)' if api_key.blank?
  if @kind == 'dataset'
    raise 'An ontology term (CHMO) is required' if @ols_term_id.blank?
  else
    raise 'A name/label is required' if @subject.blank?
  end

  response = post_messages
  raise "AI request failed (HTTP #{response.code})" unless response.code == 200

  body = JSON.parse(response.body)
  guard_finish_reason!(body)

  text = extract_text(body)
  raise 'AI returned an empty response' if text.blank?

  normalize(parse_json(text))
rescue JSON::ParserError => e
  log_ai_response('generate could not parse response', response&.body)
  Labimotion.log_exception(e)
  raise 'AI returned a response that could not be parsed as a template'
end

#ping ⇒ Object 



311
312
313
314
315
316
317
318
319
320
 
# File 'lib/labimotion/libs/ai_template.rb', line 311

def ping
  raise 'AI API key is not configured' if api_key.blank?

  started  = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  response = post_chat([{ role: 'user', content: 'ping' }], PING_MAX_TOKENS)
  ms       = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).round
  raise ping_error(response.code) unless response.code == 200

  { 'ok' => true, 'model' => model, 'endpoint' => "#{base_url}#{api_path}", 'ms' => ms }
end

#refine(current:, instruction:) ⇒ Object 



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
 
# File 'lib/labimotion/libs/ai_template.rb', line 247

def refine(current:, instruction:)
  raise 'AI API key is not configured (set config/labimotion_ai.yml :api_key or KI_TOOLBOX_API_KEY)' if api_key.blank?
  raise 'An instruction is required' if instruction.to_s.strip.blank?

  response = post_chat(refine_messages(current, instruction))
  raise "AI request failed (HTTP #{response.code})" unless response.code == 200

  body = JSON.parse(response.body)
  guard_finish_reason!(body)

  text = extract_text(body)
  raise 'AI returned an empty response' if text.blank?

  data = parse_json(text)
  result = normalize_preserving(data)
  # Guard against a malformed response silently wiping the template — a
  # dataset template always has at least one layer.
  if result['layers'].empty?
    log_ai_response('refine produced no layers', text)
    raise 'AI returned an empty template (the model response contained no layers). ' \
          'Try a more capable model.'
  end

  result['summary'] = (data.is_a?(Hash) ? data['summary'].to_s.strip : '')
  result
rescue JSON::ParserError => e
  log_ai_response('refine could not parse response', response&.body)
  Labimotion.log_exception(e)
  raise 'AI returned a response that could not be parsed as a template'
end