Class: SkillBench::Criteria

Inherits:
Object
  • Object
show all
Defined in:
lib/skill_bench/criteria.rb

Overview

Loads, validates, and represents evaluation criteria from criteria.json.

Merges eval-specific dimension overrides with built-in default descriptions and validates that dimension weights sum to exactly 100.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(path:) ⇒ Criteria

Returns a new instance of Criteria.

Parameters:

  • path (String) —

    Path to the criteria.json file.



35
36
37
 
# File 'lib/skill_bench/criteria.rb', line 35

def initialize(path:)
  @path = path
end

Instance Attribute Details

#context ⇒ Object (readonly)

Returns the value of attribute context.



11
12
13
 
# File 'lib/skill_bench/criteria.rb', line 11

def context
  @context
end

#dimensions ⇒ Object (readonly)

Returns the value of attribute dimensions.



11
12
13
 
# File 'lib/skill_bench/criteria.rb', line 11

def dimensions
  @dimensions
end

#minimum_delta ⇒ Object (readonly)

Returns the value of attribute minimum_delta.



11
12
13
 
# File 'lib/skill_bench/criteria.rb', line 11

def minimum_delta
  @minimum_delta
end

#pass_threshold ⇒ Object (readonly)

Returns the value of attribute pass_threshold.



11
12
13
 
# File 'lib/skill_bench/criteria.rb', line 11

def pass_threshold
  @pass_threshold
end

Class Method Details

.call(path:) ⇒ Hash

Loads criteria from a JSON file.

Parameters:

  • path (String) —

    Path to the criteria.json file.

Returns:

  • (Hash) —

    Service response with :success and :response keys.

Raises:

  • (TypeError) —

    when the provided path is not a string.



18
19
20
 
# File 'lib/skill_bench/criteria.rb', line 18

def self.call(path:)
  new(path:).call
end

.empty ⇒ SkillBench::Criteria

Returns an empty criteria with default thresholds and no dimensions.

Returns:



25
26
27
28
29
30
31
32
 
# File 'lib/skill_bench/criteria.rb', line 25

def self.empty
  new(path: '').tap do |criteria|
    criteria.instance_variable_set(:@context, '')
    criteria.instance_variable_set(:@pass_threshold, 70)
    criteria.instance_variable_set(:@minimum_delta, 10)
    criteria.instance_variable_set(:@dimensions, [])
  end
end

Instance Method Details

#call ⇒ Hash

Loads and validates the criteria file.

Returns:

  • (Hash) —

    Service response with criteria or error.



42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/skill_bench/criteria.rb', line 42

def call
  raw = load_json
  return raw unless raw[:success]

  data = raw[:response][:data]
  raw_dimensions = data['dimensions'] || data[:dimensions] || []
  dimensions = build_dimensions(raw_dimensions)

  core_validation = validate_core_dimensions(dimensions)
  return core_validation unless core_validation[:success]

  validation = validate_dimensions(dimensions)
  return validation unless validation[:success]

  assign_attributes(data, dimensions)

  { success: true, response: { criteria: self } }
rescue StandardError => e
  SkillBench::ErrorLogger.log_error(e, 'Criteria Load Error')
  { success: false, response: { error: { message: e.message } } }
end