Module: RubricLLM::Statistics

Defined in:
lib/rubric_llm/statistics.rb

Overview

Pure statistical helpers. No LLM calls, no state.

Class Method Summary collapse

Class Method Details

.beta_continued_fraction(a, b, x) ⇒ Object 



69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/rubric_llm/statistics.rb', line 69

def beta_continued_fraction(a, b, x)
  tiny = 1e-30
  qab = a + b
  qap = a + 1.0
  qam = a - 1.0

  c = 1.0
  d = 1.0 - ((qab * x) / qap)
  d = tiny if d.abs < tiny
  d = 1.0 / d
  fraction = d

  (1..200).each do |m|
    m2 = 2 * m

    numerator = (m * (b - m) * x) / ((qam + m2) * (a + m2))
    d = 1.0 + (numerator * d)
    d = tiny if d.abs < tiny
    c = 1.0 + (numerator / c)
    c = tiny if c.abs < tiny
    d = 1.0 / d
    fraction *= c * d

    numerator = -((a + m) * (qab + m) * x) / ((a + m2) * (qap + m2))
    d = 1.0 + (numerator * d)
    d = tiny if d.abs < tiny
    c = 1.0 + (numerator / c)
    c = tiny if c.abs < tiny
    d = 1.0 / d
    delta = c * d
    fraction *= delta

    break if (delta - 1.0).abs < 1e-12
  end

  fraction
end

.holm_adjust(p_values) ⇒ Object

Holm-Bonferroni step-down adjustment. Takes p-values in any order, returns adjusted values in the same order.



37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/rubric_llm/statistics.rb', line 37

def holm_adjust(p_values)
  count = p_values.size
  return p_values.dup if count.zero?

  adjusted = Array.new(count)
  running_max = 0.0

  p_values.each_with_index.sort_by(&:first).each_with_index do |(p_value, position), rank|
    running_max = ((count - rank) * p_value).clamp(running_max, 1.0)
    adjusted[position] = running_max
  end

  adjusted
end

.paired_t_test(scores_a, scores_b) ⇒ Object

Two-tailed paired t-test. Both arrays must already be paired and equal length. Returns 1.0 when there is nothing to test.



10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
 
# File 'lib/rubric_llm/statistics.rb', line 10

def paired_t_test(scores_a, scores_b)
  n = scores_a.size
  return 1.0 if n < 2

  diffs = scores_a.zip(scores_b).map { |x, y| y - x }
  mean_d = diffs.sum / n.to_f
  var_d = diffs.sum { |d| (d - mean_d)**2 } / (n - 1).to_f
  se = Math.sqrt(var_d / n)

  # A constant shift has no spread to test. Compare against the scale of the
  # data rather than exact zero, or float error turns it into a huge t value.
  return 1.0 if se <= Float::EPSILON * [mean_d.abs, 1.0].max

  two_tailed_p((mean_d / se).abs, n - 1)
end

.regularized_beta(x, a, b) ⇒ Object

Regularized incomplete beta function via continued fraction (Lentz's method).



53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/rubric_llm/statistics.rb', line 53

def regularized_beta(x, a, b)
  return 0.0 if x <= 0.0
  return 1.0 if x >= 1.0

  ln_beta = Math.lgamma(a + b)[0] - Math.lgamma(a)[0] - Math.lgamma(b)[0]
  front = Math.exp(ln_beta + (a * Math.log(x)) + (b * Math.log(1.0 - x)))

  result = if x < ((a + 1.0) / (a + b + 2.0))
             front * beta_continued_fraction(a, b, x) / a
           else
             1.0 - ((front * beta_continued_fraction(b, a, 1.0 - x)) / b)
           end

  result.clamp(0.0, 1.0)
end

.two_tailed_p(t, df) ⇒ Object

Two-tailed p-value for Student's t-distribution. p = I_x(df/2, 1/2) where x = df/(df + t²)



28
29
30
31
32
33
 
# File 'lib/rubric_llm/statistics.rb', line 28

def two_tailed_p(t, df)
  x = df / (df + (t**2))
  regularized_beta(x, df / 2.0, 0.5)
rescue Math::DomainError, ZeroDivisionError, FloatDomainError
  1.0
end