Class: SkillBench::DeltaReport

Inherits:
Object
  • Object
show all
Defined in:
lib/skill_bench/delta_report.rb

Overview

Computes baseline vs context deltas per dimension and determines verdict.

Verdict is true when context score meets pass_threshold AND the total delta meets minimum_delta.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(baseline:, context:, criteria:) ⇒ DeltaReport

Returns a new instance of DeltaReport.

Parameters:

  • baseline (Hash) —

    Baseline dimensions.

  • context (Hash) —

    Context dimensions.

  • criteria (SkillBench::Criteria) —

    Eval criteria.



25
26
27
28
29
30
 
# File 'lib/skill_bench/delta_report.rb', line 25

def initialize(baseline:, context:, criteria:)
  @baseline = baseline
  @context = context
  @criteria = criteria
  @deltas = {}
end

Instance Attribute Details

#baseline_dimensions ⇒ Object (readonly)

Returns the value of attribute baseline_dimensions.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def baseline_dimensions
  @baseline_dimensions
end

#baseline_scores ⇒ Object (readonly)

Returns the value of attribute baseline_scores.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def baseline_scores
  @baseline_scores
end

#baseline_total ⇒ Object (readonly)

Returns the value of attribute baseline_total.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def baseline_total
  @baseline_total
end

#context_dimensions ⇒ Object (readonly)

Returns the value of attribute context_dimensions.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def context_dimensions
  @context_dimensions
end

#context_scores ⇒ Object (readonly)

Returns the value of attribute context_scores.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def context_scores
  @context_scores
end

#context_total ⇒ Object (readonly)

Returns the value of attribute context_total.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def context_total
  @context_total
end

#criteria ⇒ Object (readonly)

Returns the value of attribute criteria.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def criteria
  @criteria
end

#deltas ⇒ Object (readonly)

Returns the value of attribute deltas.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def deltas
  @deltas
end

#verdict ⇒ Object (readonly)

Returns the value of attribute verdict.



9
10
11
 
# File 'lib/skill_bench/delta_report.rb', line 9

def verdict
  @verdict
end

Class Method Details

.call(baseline:, context:, criteria:) ⇒ Hash

Computes deltas and verdict from baseline and context judge responses.

Parameters:

  • baseline (Hash) —

    Baseline judge dimensions hash.

  • context (Hash) —

    Context judge dimensions hash.

  • criteria (SkillBench::Criteria) —

    The eval criteria with thresholds.

Returns:

  • (Hash) —

    Service response with delta_report or error.



18
19
20
 
# File 'lib/skill_bench/delta_report.rb', line 18

def self.call(baseline:, context:, criteria:)
  new(baseline:, context:, criteria:).call
end

Instance Method Details

#call ⇒ Hash

Computes deltas and determines verdict.

Returns:

  • (Hash) —

    Service response with delta_report or error.



35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/skill_bench/delta_report.rb', line 35

def call
  return mismatch_result unless dimensions_match?

  @baseline_dimensions = deep_copy_dimensions(baseline)
  @context_dimensions = deep_copy_dimensions(context)
  @baseline_scores = extract_scores(baseline)
  @context_scores = extract_scores(context)
  compute_totals
  compute_deltas
  determine_verdict

  { success: true, response: { delta_report: self } }
rescue StandardError => e
  SkillBench::ErrorLogger.log_error(e, 'DeltaReport Error')
  { success: false, response: { error: { message: e.message } } }
end