Class: SkiftetStatistical::Policies::UCB1

Inherits:
Base
  • Object
show all
Defined in:
lib/skiftet_statistical/policies/ucb1.rb

Overview

UCB1: deterministic optimism under uncertainty. Pull the arm maximising mean + sqrt(c * ln(total_pulls) / arm_pulls). Each arm is pulled once first (the confidence bound is undefined at zero pulls). Larger c explores more; c = 2.0 is the classic Auer et al. value.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(c: 2.0, rng: Random.new) ⇒ UCB1

Returns a new instance of UCB1.

Raises:

  • (ArgumentError) 


12
13
14
15
16
17
18
 
# File 'lib/skiftet_statistical/policies/ucb1.rb', line 12

def initialize(c: 2.0, rng: Random.new)
  super()
  raise ArgumentError, "c must be > 0" unless c.positive?

  @c = Float(c)
  @rng = rng
end

Instance Attribute Details

#c ⇒ Object (readonly)

Returns the value of attribute c.



10
11
12
 
# File 'lib/skiftet_statistical/policies/ucb1.rb', line 10

def c
  @c
end

Instance Method Details

#choose(arms) ⇒ Object 



20
21
22
23
24
25
26
27
28
29
30
31
32
 
# File 'lib/skiftet_statistical/policies/ucb1.rb', line 20

def choose(arms)
  ensure_arms!(arms)

  fresh = unpulled(arms)
  return fresh[@rng.rand(fresh.length)] unless fresh.empty?

  ln_total = Math.log(arms.sum(&:pulls))
  scored = arms.map do |arm|
    bonus = Math.sqrt(@c * ln_total / arm.pulls)
    [ arm, arm.mean + bonus ]
  end
  pick_max(scored, @rng)
end

#to_h ⇒ Object 



34
35
36
 
# File 'lib/skiftet_statistical/policies/ucb1.rb', line 34

def to_h
  super.merge(c: @c)
end