Class: SkiftetStatistical::Arm

Inherits:
Object
  • Object
show all
Defined in:
lib/skiftet_statistical/arm.rb

Overview

One option ("arm") the bandit can choose, tracking online reward statistics.

Rewards are expected in [0.0, 1.0] for the Bernoulli/Beta policies (Thompson Sampling, UCB1, Epsilon-Greedy treat the mean as a success rate). A binary 0/1 reward is the common case ("did this share convert?"), but any value in [0, 1] works (the summed rewards act as fractional successes).

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(name, pulls: 0, reward_sum: 0.0, reward_square_sum: 0.0) ⇒ Arm

Returns a new instance of Arm.

Raises:

  • (ArgumentError) 


13
14
15
16
17
18
19
20
 
# File 'lib/skiftet_statistical/arm.rb', line 13

def initialize(name, pulls: 0, reward_sum: 0.0, reward_square_sum: 0.0)
  raise ArgumentError, "arm name cannot be nil" if name.nil?

  @name = name
  @pulls = Integer(pulls)
  @reward_sum = Float(reward_sum)
  @reward_square_sum = Float(reward_square_sum)
end

Instance Attribute Details

#name ⇒ Object (readonly)

Returns the value of attribute name.



11
12
13
 
# File 'lib/skiftet_statistical/arm.rb', line 11

def name
  @name
end

#pulls ⇒ Object (readonly)

Returns the value of attribute pulls.



11
12
13
 
# File 'lib/skiftet_statistical/arm.rb', line 11

def pulls
  @pulls
end

#reward_square_sum ⇒ Object (readonly)

Returns the value of attribute reward_square_sum.



11
12
13
 
# File 'lib/skiftet_statistical/arm.rb', line 11

def reward_square_sum
  @reward_square_sum
end

#reward_sum ⇒ Object (readonly)

Returns the value of attribute reward_sum.



11
12
13
 
# File 'lib/skiftet_statistical/arm.rb', line 11

def reward_sum
  @reward_sum
end

Class Method Details

.from_h(hash) ⇒ Object 



70
71
72
73
74
75
76
77
78
 
# File 'lib/skiftet_statistical/arm.rb', line 70

def self.from_h(hash)
  h = hash.transform_keys(&:to_sym)
  new(
    h.fetch(:name),
    pulls: h.fetch(:pulls, 0),
    reward_sum: h.fetch(:reward_sum, 0.0),
    reward_square_sum: h.fetch(:reward_square_sum, 0.0),
  )
end

Instance Method Details

#failures ⇒ Object 



53
54
55
 
# File 'lib/skiftet_statistical/arm.rb', line 53

def failures
  [ @pulls - @reward_sum, 0.0 ].max
end

#mean ⇒ Object Also known as: rate

Empirical mean reward (0.0 when never pulled).



32
33
34
35
36
 
# File 'lib/skiftet_statistical/arm.rb', line 32

def mean
  return 0.0 if @pulls.zero?

  @reward_sum / @pulls
end

#pulled? ⇒ Boolean

Returns:

  • (Boolean) 


57
58
59
 
# File 'lib/skiftet_statistical/arm.rb', line 57

def pulled?
  @pulls.positive?
end

#successes ⇒ Object

Beta-Bernoulli view: summed rewards are "successes", the remaining pulls "failures". With [0, 1] rewards these can be fractional — Beta handles that.



49
50
51
 
# File 'lib/skiftet_statistical/arm.rb', line 49

def successes
  @reward_sum
end

#to_h ⇒ Object 



61
62
63
64
65
66
67
68
 
# File 'lib/skiftet_statistical/arm.rb', line 61

def to_h
  {
    name: @name,
    pulls: @pulls,
    reward_sum: @reward_sum,
    reward_square_sum: @reward_square_sum
  }
end

#update(reward) ⇒ Object

Record one observed reward for this arm. Returns self for chaining.



23
24
25
26
27
28
29
 
# File 'lib/skiftet_statistical/arm.rb', line 23

def update(reward)
  r = Float(reward)
  @pulls += 1
  @reward_sum += r
  @reward_square_sum += r * r
  self
end

#variance ⇒ Object

Population variance of observed rewards (0.0 with fewer than two pulls).



40
41
42
43
44
45
 
# File 'lib/skiftet_statistical/arm.rb', line 40

def variance
  return 0.0 if @pulls < 2

  m = mean
  [ (@reward_square_sum / @pulls) - (m * m), 0.0 ].max
end