Class: LlmConductor::Eval::ReportBuilder

Inherits:
Object
  • Object
show all
Defined in:
lib/llm_conductor/eval/report_builder.rb

Overview

Pure aggregation: turns a run’s per-(input, model) rows into a Report (CSV string + decision-aid markdown + per-model summary + needs-review list). Unlike the Rails prototype’s ReportBuilder it writes no files; persistence is the caller’s / Store’s job.

rows is an Array of { model_result: Result, judge_verdict: Verdict }.

Constant Summary collapse

BASE_CSV_COLUMNS = 
%w[
  input_id input_label model vendor status
  latency_ms input_tokens output_tokens total_tokens estimated_cost_usd
  parsed_score parsed_bucket
  judge_quality_score
].freeze
JUDGE_TAIL_COLUMNS = 
%w[
  judge_verdict_one_line judge_issues judge_error
  self_judge needs_human_review review_reasons
  raw_output_ref parsed_output_ref error
].freeze

Instance Method Summary collapse

Constructor Details

#initialize(rows:, run_id:, judge_model:, spec:) ⇒ ReportBuilder

Returns a new instance of ReportBuilder.



29
30
31
32
33
34
 
# File 'lib/llm_conductor/eval/report_builder.rb', line 29

def initialize(rows:, run_id:, judge_model:, spec:)
  @rows = rows
  @run_id = run_id
  @judge_model = judge_model
  @spec = spec
end

Instance Method Details

#build ⇒ Object 



36
37
38
39
40
41
42
43
44
45
46
 
# File 'lib/llm_conductor/eval/report_builder.rb', line 36

def build
  bucket_disagreement = compute_bucket_disagreement
  summary = build_summary
  Report.new(
    rows: @rows,
    summary:,
    needs_review: build_needs_review(bucket_disagreement),
    csv_string: build_csv(bucket_disagreement),
    markdown_string: build_markdown(bucket_disagreement, summary)
  )
end