Class: SkiftetStatistical::Policies::Softmax

Inherits:
Base
  • Object
show all
Defined in:
lib/skiftet_statistical/policies/softmax.rb

Overview

Softmax / Boltzmann exploration: pick arm i with probability proportional to exp(mean_i / temperature). Low temperature => near-greedy; high temperature => near-uniform exploration.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(temperature: 0.1, rng: Random.new) ⇒ Softmax

Returns a new instance of Softmax.

Raises:

  • (ArgumentError) 


11
12
13
14
15
16
17
 
# File 'lib/skiftet_statistical/policies/softmax.rb', line 11

def initialize(temperature: 0.1, rng: Random.new)
  super()
  raise ArgumentError, "temperature must be > 0" unless temperature.positive?

  @temperature = Float(temperature)
  @rng = rng
end

Instance Attribute Details

#temperature ⇒ Object (readonly)

Returns the value of attribute temperature.



9
10
11
 
# File 'lib/skiftet_statistical/policies/softmax.rb', line 9

def temperature
  @temperature
end

Instance Method Details

#choose(arms) ⇒ Object 



19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
 
# File 'lib/skiftet_statistical/policies/softmax.rb', line 19

def choose(arms)
  ensure_arms!(arms)

  # Shift by the max mean for numerical stability (exp can overflow).
  max_mean = arms.map(&:mean).max
  weights = arms.map { |a| Math.exp((a.mean - max_mean) / @temperature) }
  total = weights.sum
  target = @rng.rand * total

  cumulative = 0.0
  arms.each_with_index do |arm, i|
    cumulative += weights[i]
    return arm if cumulative >= target
  end
  arms.last
end

#to_h ⇒ Object 



36
37
38
 
# File 'lib/skiftet_statistical/policies/softmax.rb', line 36

def to_h
  super.merge(temperature: @temperature)
end