Class: ClaudeMemory::Dashboard::Trust

Inherits:
Object
  • Object
show all
Defined in:
lib/claude_memory/dashboard/trust.rb

Overview

Sidebar data for the feed-first dashboard. Six surfaces, each answering a different “is memory helping/costing/clean?” question:

  1. Moments this week + week-over-week delta — the headline value number. A moment is any meaningful activity event (recall hit, extraction, context injection, conflict detected). Ingest-only events don’t count because they’re not directly user-visible value.

  2. “What memory knows about you” — up to 5 global facts rendered as plain English. The trust panel’s most compelling surface: users can sanity-check what’s being injected into their sessions.

  3. Needs review — open conflicts plus stale facts (active but never recalled in the last N days) plus empty recalls (queries that returned nothing). A single actionable count; the feed surfaces the individual items.

  4. Utilization (30d) — of facts extracted in the last 30 days, how many has Claude actually surfaced via recall or context injection. Low ratios are a signal too: memory accumulating knowledge that Claude isn’t reaching for.

  5. Token budget (30d, 0.11.0+) — p50/p95/avg ‘context_tokens` injected per SessionStart. Answers “what does memory cost per session?” via numbers a skeptical user can read.

  6. Quality score (live + historical, 0.11.0+) — hallucination-rate proxy: 100 - (suspect_pct + bare_pct), clamped 0..100. Live is over the last UTILIZATION_DAYS; historical mirrors the same calculation across all active facts as a supplementary baseline. See ‘quality_review.md` 2026-04-30 note for why the split exists.

Constant Summary collapse

WEEK_SECONDS = 
7 * 86_400
UTILIZATION_DAYS = 
30
VALUE_EVENT_TYPES = 
%w[hook_context recall store_extraction].freeze

Instance Method Summary collapse

Constructor Details

#initialize(manager) ⇒ Trust

Returns a new instance of Trust.



41
42
43
 
# File 'lib/claude_memory/dashboard/trust.rb', line 41

def initialize(manager)
  @manager = manager
end

Instance Method Details

#aggregate_quality_counts(cutoff: nil) ⇒ Object 



134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
 
# File 'lib/claude_memory/dashboard/trust.rb', line 134

def aggregate_quality_counts(cutoff: nil)
  detector = Distill::BareConclusionDetector.new
  suspect = 0
  bare = 0
  total = 0

  %w[project global].each do |scope|
    store = @manager.store_if_exists(scope)
    next unless store
    dataset = store.facts.where(status: "active")
    dataset = dataset.where { created_at >= cutoff } if cutoff
    total += dataset.count
    suspect += dataset.where(predicate: "reference").count
    dataset.where(predicate: %w[decision convention])
      .select(:predicate, :object_literal)
      .all
      .each { |row| bare += 1 if detector.bare_conclusion?(row) }
  end

  {total_active: total, suspect_count: suspect, bare_conclusion_count: bare}
end

#compute_quality(cutoff:) ⇒ Object 



112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/claude_memory/dashboard/trust.rb', line 112

def compute_quality(cutoff:)
  breakdown = aggregate_quality_counts(cutoff: cutoff)
  total = breakdown[:total_active]

  return zero_breakdown if total.zero?

  suspect_pct = (breakdown[:suspect_count] * 100.0 / total).round(1)
  bare_pct = (breakdown[:bare_conclusion_count] * 100.0 / total).round(1)
  score = (100 - (suspect_pct + bare_pct)).clamp(0, 100).round

  breakdown.merge(
    suspect_pct: suspect_pct,
    bare_pct: bare_pct,
    score: score
  )
end

#percentile(sorted, pct) ⇒ Object 



199
200
201
202
203
204
205
 
# File 'lib/claude_memory/dashboard/trust.rb', line 199

def percentile(sorted, pct)
  return 0 if sorted.empty?
  idx = (sorted.size * pct).ceil - 1
  idx = 0 if idx < 0
  idx = sorted.size - 1 if idx >= sorted.size
  sorted[idx]
end

#quality_score ⇒ Object

The trust panel’s hallucination-rate proxy. Counts two pollution signals:

- suspect: facts that ReferenceMaterialDetector retagged from
  `convention` to `reference` predicate (descriptions of external
  projects mislabeled as user conventions).
- bare_conclusion: `decision` / `convention` facts whose object
  skipped the prompt-mandated reason clause and so are dead
  weight once the originating context is gone.

Reports two windows so users can distinguish historical noise from live extraction quality (per ‘quality_review.md` 2026-04-30 investigation): the headline `score` is computed over facts created within the last UTILIZATION_DAYS — that’s the actionable signal. The ‘historical` block reports the same counts over all active facts so legacy data is visible without dominating.

Score = 100 - (suspect_pct + bare_pct), clamped 0..100. Lower is worse. Returns 100 (perfect) when there are no facts in the window so a quiet week isn’t penalized.



77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/claude_memory/dashboard/trust.rb', line 77

def quality_score
  cutoff = (Time.now.utc - UTILIZATION_DAYS * 86_400).iso8601
  live = compute_quality(cutoff: cutoff)
  historical = compute_quality(cutoff: nil)

  live.merge(
    window_days: UTILIZATION_DAYS,
    historical: historical
  )
rescue Sequel::DatabaseError => e
  ClaudeMemory.logger.debug("Trust#quality_score failed: #{e.message}")
  quality_score_zero
end

#quality_score_zero ⇒ Object 



92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/claude_memory/dashboard/trust.rb', line 92

def quality_score_zero
  {
    total_active: 0,
    suspect_count: 0,
    bare_conclusion_count: 0,
    suspect_pct: 0.0,
    bare_pct: 0.0,
    score: 100,
    window_days: UTILIZATION_DAYS,
    historical: {
      total_active: 0,
      suspect_count: 0,
      bare_conclusion_count: 0,
      suspect_pct: 0.0,
      bare_pct: 0.0,
      score: 100
    }
  }
end

#snapshot ⇒ Object 



45
46
47
48
49
50
51
52
53
54
55
 
# File 'lib/claude_memory/dashboard/trust.rb', line 45

def snapshot
  {
    weekly_moments: weekly_moments,
    fingerprint: fingerprint,
    needs_review: needs_review,
    utilization: utilization,
    feedback: feedback_summary,
    token_budget: token_budget,
    quality_score: quality_score
  }
end

#token_budget ⇒ Object

What does memory cost? Aggregates ‘context_tokens` from successful `hook_context` activity events over the last UTILIZATION_DAYS so a skeptical user can see the per-session token cost in p50/p95.

Shape: p95:, avg:, sample_size:, window_days: All ints. Returns zeros when there are no events in the window.



162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
 
# File 'lib/claude_memory/dashboard/trust.rb', line 162

def token_budget
  store = @manager.default_store(prefer: :project)
  return token_budget_zero unless store

  cutoff = (Time.now.utc - UTILIZATION_DAYS * 86_400).iso8601
  rows = store.activity_events
    .where(event_type: "hook_context", status: "success")
    .where { occurred_at >= cutoff }
    .select(:detail_json)
    .all

  tokens = rows.filter_map do |row|
    details = row[:detail_json] ? JSON.parse(row[:detail_json]) : {}
    value = details["context_tokens"]
    value if value.is_a?(Integer) && value > 0
  end

  return token_budget_zero if tokens.empty?

  sorted = tokens.sort
  {
    p50: percentile(sorted, 0.50),
    p95: percentile(sorted, 0.95),
    avg: (sorted.sum.to_f / sorted.size).round,
    sample_size: sorted.size,
    window_days: UTILIZATION_DAYS
  }
rescue Sequel::DatabaseError, JSON::ParserError => e
  ClaudeMemory.logger.debug("Trust#token_budget failed: #{e.message}")
  token_budget_zero
end

#token_budget_zero ⇒ Object 



195
196
197
 
# File 'lib/claude_memory/dashboard/trust.rb', line 195

def token_budget_zero
  {p50: 0, p95: 0, avg: 0, sample_size: 0, window_days: UTILIZATION_DAYS}
end

#utilization ⇒ Object

The ROI signal: of the facts Claude has extracted into memory over the last UTILIZATION_DAYS, how many has Claude actually used (appeared in any recall or context injection’s top_fact_ids)? Low ratios are themselves a signal — it means memory is accumulating knowledge but Claude isn’t reaching for it. Anomalies worth surfacing honestly.

Shape: Int, used: Int, ratio_pct: Int, window_days: Int Both counts are scope-union (project + global) so the headline number reflects everything memory did, not just one store.



372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
 
# File 'lib/claude_memory/dashboard/trust.rb', line 372

def utilization
  cutoff = (Time.now.utc - UTILIZATION_DAYS * 86_400).iso8601
  extracted_pairs = extracted_fact_pairs(cutoff)
  used_pairs = used_fact_pairs(cutoff)

  extracted = extracted_pairs.size
  # "Used" counted against the extracted set — a fact used but not
  # extracted in this window (taught earlier, used now) is still
  # re-use worth recognizing; count it too.
  used_from_extracted = (used_pairs & extracted_pairs).size
  used_total = used_pairs.size

  ratio_pct = extracted.zero? ? 0 : ((used_from_extracted.to_f / extracted) * 100).round

  {
    extracted: extracted,
    used: used_total,
    used_from_extracted: used_from_extracted,
    ratio_pct: ratio_pct,
    window_days: UTILIZATION_DAYS
  }
rescue Sequel::DatabaseError, JSON::ParserError => e
  ClaudeMemory.logger.debug("Trust#utilization failed: #{e.message}")
  {extracted: 0, used: 0, used_from_extracted: 0, ratio_pct: 0, window_days: UTILIZATION_DAYS}
end

#zero_breakdown ⇒ Object 



129
130
131
132
 
# File 'lib/claude_memory/dashboard/trust.rb', line 129

def zero_breakdown
  {total_active: 0, suspect_count: 0, bare_conclusion_count: 0,
   suspect_pct: 0.0, bare_pct: 0.0, score: 100}
end