Class: RailsErrorDashboard::Queries::LlmHealthSummary

Inherits:
Object
  • Object
show all
Defined in:
lib/rails_error_dashboard/queries/llm_health_summary.rb

Overview

Query: Aggregate LLM call breadcrumbs across all errors, grouped by “provider · model”. Scans error_logs breadcrumbs JSON, filters for “llm” and “llm_tool” category crumbs, and computes per-model stats: call count, tool count, avg tokens, avg latency, error rate, cost, top error class.

Sorted by error rate desc, then unique-error count desc, then call volume desc — so the model causing the most errors floats to the top.

Cost is read straight from the breadcrumb metadata (already estimated at capture time by LlmCallSubscriber via LlmCostEstimator). No re-estimation.

Constant Summary collapse

DANGER_THRESHOLD =

error rate %

10.0
WARNING_THRESHOLD = 
5.0

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(days = 30, application_id: nil) ⇒ LlmHealthSummary

Returns a new instance of LlmHealthSummary.



36
37
38
39
40
 
# File 'lib/rails_error_dashboard/queries/llm_health_summary.rb', line 36

def initialize(days = 30, application_id: nil)
  @days = days
  @application_id = application_id
  @start_date = days.days.ago
end

Class Method Details

.blank_totals ⇒ Object

Public helper so controllers can render an empty-state shell without running the query (e.g., when the feature is disabled).



25
26
27
28
29
30
31
32
33
34
 
# File 'lib/rails_error_dashboard/queries/llm_health_summary.rb', line 25

def self.blank_totals
  {
    total_calls: 0,
    total_tool_calls: 0,
    model_count: 0,
    unique_error_count: 0,
    error_rate: 0.0,
    total_cost_usd: 0.0
  }
end

.call(days = 30, application_id: nil) ⇒ Object 



19
20
21
 
# File 'lib/rails_error_dashboard/queries/llm_health_summary.rb', line 19

def self.call(days = 30, application_id: nil)
  new(days, application_id: application_id).call
end

Instance Method Details

#call ⇒ Object 



42
43
44
45
46
47
48
 
# File 'lib/rails_error_dashboard/queries/llm_health_summary.rb', line 42

def call
  models = aggregated_models
  {
    models: models,
    totals: totals_for(models)
  }
end