Class: SkillBench::Judge::Response

Inherits:
Object
  • Object
show all
Defined in:
lib/skill_bench/judge/response.rb

Overview

Parses and validates structured JSON responses from the LLM judge.

Expects a JSON object with a ‘dimensions’ key mapping dimension names to score hashes, and an optional ‘overall_reasoning’ string.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(json:) ⇒ Response

Returns a new instance of Response.

Parameters:

  • json (String) —

    The raw JSON string from the judge.



23
24
25
 
# File 'lib/skill_bench/judge/response.rb', line 23

def initialize(json:)
  @json = json
end

Instance Attribute Details

#dimensions ⇒ Object (readonly)

Returns the value of attribute dimensions.



12
13
14
 
# File 'lib/skill_bench/judge/response.rb', line 12

def dimensions
  @dimensions
end

#overall_reasoning ⇒ Object (readonly)

Returns the value of attribute overall_reasoning.



12
13
14
 
# File 'lib/skill_bench/judge/response.rb', line 12

def overall_reasoning
  @overall_reasoning
end

Class Method Details

.call(json:) ⇒ Hash

Parses a judge JSON string.

Parameters:

  • json (String) —

    The raw JSON string from the judge.

Returns:

  • (Hash) —

    Service response with parsed judge response or error.



18
19
20
 
# File 'lib/skill_bench/judge/response.rb', line 18

def self.call(json:)
  new(json:).call
end

Instance Method Details

#call ⇒ Hash

Parses and validates the judge JSON.

Returns:

  • (Hash) —

    Service response with judge response or error.



30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
 
# File 'lib/skill_bench/judge/response.rb', line 30

def call
  data = parse_json
  return data unless data[:success]

  payload = data[:response][:data]
  validation = validate_structure(payload)
  return validation unless validation[:success]

  dims = payload['dimensions'] || payload[:dimensions]
  extracted = extract_dimensions(dims)
  return extracted unless extracted[:success]

  @dimensions = extracted[:response][:dimensions]
  @overall_reasoning = payload['overall_reasoning'] || payload[:overall_reasoning] || ''

  { success: true, response: { judge_response: self } }
rescue StandardError => e
  SkillBench::ErrorLogger.log_error(e, 'Judge::Response Parse Error')
  { success: false, response: { error: { message: e.message } } }
end