Class: SkiftetStatistical::Policies::EpsilonGreedy

Inherits:
Base
  • Object
show all
Defined in:
lib/skiftet_statistical/policies/epsilon_greedy.rb

Overview

epsilon-greedy: exploit the best-mean arm with probability (1 - epsilon), explore a uniformly random arm with probability epsilon. Every arm is pulled once first so none is starved by a zero initial mean.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(epsilon: 0.1, rng: Random.new) ⇒ EpsilonGreedy

Returns a new instance of EpsilonGreedy.

Raises:

  • (ArgumentError) 


11
12
13
14
15
16
17
 
# File 'lib/skiftet_statistical/policies/epsilon_greedy.rb', line 11

def initialize(epsilon: 0.1, rng: Random.new)
  super()
  raise ArgumentError, "epsilon must be in [0, 1]" unless (0.0..1.0).cover?(epsilon)

  @epsilon = Float(epsilon)
  @rng = rng
end

Instance Attribute Details

#epsilon ⇒ Object (readonly)

Returns the value of attribute epsilon.



9
10
11
 
# File 'lib/skiftet_statistical/policies/epsilon_greedy.rb', line 9

def epsilon
  @epsilon
end

Instance Method Details

#choose(arms) ⇒ Object 



19
20
21
22
23
24
25
26
27
28
29
30
 
# File 'lib/skiftet_statistical/policies/epsilon_greedy.rb', line 19

def choose(arms)
  ensure_arms!(arms)

  fresh = unpulled(arms)
  return fresh[@rng.rand(fresh.length)] unless fresh.empty?

  if @rng.rand < @epsilon
    arms[@rng.rand(arms.length)]
  else
    pick_max(arms.map { |a| [ a, a.mean ] }, @rng)
  end
end

#to_h ⇒ Object 



32
33
34
 
# File 'lib/skiftet_statistical/policies/epsilon_greedy.rb', line 32

def to_h
  super.merge(epsilon: @epsilon)
end