Class: LlmConductor::Eval::Store::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/llm_conductor/eval/store/base.rb

Overview

Pluggable persistence interface for an eval run. Replaces the prototype’s hard-coded Rails.root.join(‘tmp’, …) + File.read/write calls.

Two implementations ship with the gem: InMemory (default; nothing hits disk) and FileStore (resumable, reproduces the prototype’s tmp/<run_id>/ layout). Implement this interface to persist anywhere else.

Write methods return an opaque “ref” (a filesystem path for FileStore, a key for InMemory) recorded on the Result for the report’s path columns.

Direct Known Subclasses

FileStore, InMemory

Instance Method Summary collapse

Instance Method Details

#completed?(_run_id, _input_id, _model_slug) ⇒ Boolean

True when this (input, model) pair already has stored output — lets a future restart skip already-completed pairs.

Returns:

  • (Boolean)

Raises:

  • (NotImplementedError) 


52
53
54
 
# File 'lib/llm_conductor/eval/store/base.rb', line 52

def completed?(_run_id, _input_id, _model_slug)
  raise NotImplementedError
end

#read_input_data(_run_id, _input_id) ⇒ Object

Enables self-contained re-judge / report without the original inputs.

Raises:

  • (NotImplementedError) 


38
39
40
 
# File 'lib/llm_conductor/eval/store/base.rb', line 38

def read_input_data(_run_id, _input_id)
  raise NotImplementedError
end

#read_manifest(_run_id) ⇒ Object

Raises:

  • (NotImplementedError) 


46
47
48
 
# File 'lib/llm_conductor/eval/store/base.rb', line 46

def read_manifest(_run_id)
  raise NotImplementedError
end

#read_parsed(_run_id, _input_id, _model_slug) ⇒ Object

Returns the parsed Hash/Array (not the ref), or nil if absent.

Raises:

  • (NotImplementedError) 


29
30
31
 
# File 'lib/llm_conductor/eval/store/base.rb', line 29

def read_parsed(_run_id, _input_id, _model_slug)
  raise NotImplementedError
end

#read_raw(_run_id, _input_id, _model_slug) ⇒ Object

Raises:

  • (NotImplementedError) 


20
21
22
 
# File 'lib/llm_conductor/eval/store/base.rb', line 20

def read_raw(_run_id, _input_id, _model_slug)
  raise NotImplementedError
end

#write_input_data(_run_id, _input_id, _hash) ⇒ Object

Raises:

  • (NotImplementedError) 


33
34
35
 
# File 'lib/llm_conductor/eval/store/base.rb', line 33

def write_input_data(_run_id, _input_id, _hash)
  raise NotImplementedError
end

#write_manifest(_run_id, _manifest_hash) ⇒ Object

Raises:

  • (NotImplementedError) 


42
43
44
 
# File 'lib/llm_conductor/eval/store/base.rb', line 42

def write_manifest(_run_id, _manifest_hash)
  raise NotImplementedError
end

#write_parsed(_run_id, _input_id, _model_slug, _hash) ⇒ Object

Raises:

  • (NotImplementedError) 


24
25
26
 
# File 'lib/llm_conductor/eval/store/base.rb', line 24

def write_parsed(_run_id, _input_id, _model_slug, _hash)
  raise NotImplementedError
end

#write_raw(_run_id, _input_id, _model_slug, _text) ⇒ Object

Raises:

  • (NotImplementedError) 


16
17
18
 
# File 'lib/llm_conductor/eval/store/base.rb', line 16

def write_raw(_run_id, _input_id, _model_slug, _text)
  raise NotImplementedError
end