Module: ClaudeMemory::Store::MetricsAggregator

Included in:
SQLiteStore
Defined in:
lib/claude_memory/store/metrics_aggregator.rb

Overview

Ingestion metrics persistence and aggregation for the SQLiteStore. Records per-distillation LLM token usage and extraction counts, and computes totals + efficiency ratios over the full history.

Instance Method Summary collapse

Instance Method Details

#aggregate_ingestion_metrics ⇒ Hash?

Compute aggregate ingestion metrics across all distillation runs.

Returns:

  • (Hash, nil) —

    totals and efficiency ratio, or nil if no data



38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/claude_memory/store/metrics_aggregator.rb', line 38

def aggregate_ingestion_metrics
  # standard:disable Performance/Detect (Sequel DSL requires .select{}.first)
  result = ingestion_metrics
    .select {
      [
        sum(:input_tokens).as(:total_input),
        sum(:output_tokens).as(:total_output),
        sum(:facts_extracted).as(:total_facts),
        count(:id).as(:total_ops)
      ]
    }
    .first
  # standard:enable Performance/Detect

  return nil if result.nil? || result[:total_ops].to_i.zero?

  total_input = result[:total_input].to_i
  total_output = result[:total_output].to_i
  total_facts = result[:total_facts].to_i
  total_ops = result[:total_ops].to_i

  efficiency = total_input.zero? ? 0.0 : (total_facts.to_f / total_input * 1000).round(2)

  {
    total_input_tokens: total_input,
    total_output_tokens: total_output,
    total_facts_extracted: total_facts,
    total_operations: total_ops,
    avg_facts_per_1k_input_tokens: efficiency
  }
end

#backfill_distillation_metrics! ⇒ Integer

Mark all undistilled content items as distilled with zero token counts. Used for backfilling legacy content that predates the metrics table.

Returns:

  • (Integer) —

    number of items backfilled



73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
 
# File 'lib/claude_memory/store/metrics_aggregator.rb', line 73

def backfill_distillation_metrics!
  undistilled_ids = content_items
    .left_join(:ingestion_metrics, content_item_id: :id)
    .where(Sequel[:ingestion_metrics][:id] => nil)
    .select_map(Sequel[:content_items][:id])

  return 0 if undistilled_ids.empty?

  now = Time.now.utc.iso8601
  undistilled_ids.each do |cid|
    ingestion_metrics.insert(
      content_item_id: cid,
      input_tokens: 0,
      output_tokens: 0,
      facts_extracted: 0,
      created_at: now
    )
  end

  undistilled_ids.size
end

#count_undistilled(min_length: 200) ⇒ Integer

Count content items that have not yet been distilled.

Parameters:

  • min_length (Integer) (defaults to: 200) —

    minimum byte_len threshold

Returns:

  • (Integer) 


12
13
14
15
16
17
18
 
# File 'lib/claude_memory/store/metrics_aggregator.rb', line 12

def count_undistilled(min_length: 200)
  content_items
    .left_join(:ingestion_metrics, content_item_id: :id)
    .where(Sequel[:ingestion_metrics][:id] => nil)
    .where { byte_len >= min_length }
    .count
end

#record_ingestion_metrics(content_item_id:, input_tokens:, output_tokens:, facts_extracted:) ⇒ Integer

Record token usage and extraction counts for a distillation run.

Parameters:

  • content_item_id (Integer) —

    content item that was distilled

  • input_tokens (Integer) —

    LLM input tokens consumed

  • output_tokens (Integer) —

    LLM output tokens consumed

  • facts_extracted (Integer) —

    number of facts extracted

Returns:

  • (Integer) —

    inserted row id



26
27
28
29
30
31
32
33
34
 
# File 'lib/claude_memory/store/metrics_aggregator.rb', line 26

def record_ingestion_metrics(content_item_id:, input_tokens:, output_tokens:, facts_extracted:)
  ingestion_metrics.insert(
    content_item_id: content_item_id,
    input_tokens: input_tokens,
    output_tokens: output_tokens,
    facts_extracted: facts_extracted,
    created_at: Time.now.utc.iso8601
  )
end