Class: SkiftetStatistical::Policies::ThompsonSampling

Inherits:
Base
  • Object
show all
Defined in:
lib/skiftet_statistical/policies/thompson_sampling.rb

Overview

Thompson Sampling (Beta-Bernoulli). For each arm draw theta ~ Beta(alpha0 + successes, beta0 + failures) and pull the arm with the highest draw. It balances exploration and exploitation automatically: under-sampled arms have wide posteriors and get tried often, while the best arm is chosen more and more as evidence accrues. With no data every arm is Beta(1, 1) = uniform, so the opening pulls are pure (random) exploration.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(prior_alpha: 1.0, prior_beta: 1.0, rng: Random.new) ⇒ ThompsonSampling

Returns a new instance of ThompsonSampling.



14
15
16
17
18
19
20
 
# File 'lib/skiftet_statistical/policies/thompson_sampling.rb', line 14

def initialize(prior_alpha: 1.0, prior_beta: 1.0, rng: Random.new)
  super()
  @prior_alpha = Float(prior_alpha)
  @prior_beta = Float(prior_beta)
  @sampler = Sampler.new(rng)
  @rng = rng
end

Instance Attribute Details

#prior_alpha ⇒ Object (readonly)

Returns the value of attribute prior_alpha.



12
13
14
 
# File 'lib/skiftet_statistical/policies/thompson_sampling.rb', line 12

def prior_alpha
  @prior_alpha
end

#prior_beta ⇒ Object (readonly)

Returns the value of attribute prior_beta.



12
13
14
 
# File 'lib/skiftet_statistical/policies/thompson_sampling.rb', line 12

def prior_beta
  @prior_beta
end

Instance Method Details

#choose(arms) ⇒ Object 



22
23
24
25
26
27
28
29
30
 
# File 'lib/skiftet_statistical/policies/thompson_sampling.rb', line 22

def choose(arms)
  ensure_arms!(arms)

  scored = arms.map do |arm|
    theta = @sampler.beta(@prior_alpha + arm.successes, @prior_beta + arm.failures)
    [ arm, theta ]
  end
  pick_max(scored, @rng)
end

#to_h ⇒ Object 



32
33
34
 
# File 'lib/skiftet_statistical/policies/thompson_sampling.rb', line 32

def to_h
  super.merge(prior_alpha: @prior_alpha, prior_beta: @prior_beta)
end