Class: SkiftetStatistical::Bandit

Inherits:
Object
  • Object
show all
Defined in:
lib/skiftet_statistical/bandit.rb

Overview

The bandit: a named set of arms plus a selection policy. Ask it which arm to play (#select), observe a reward, and tell it (#record). All state lives in the arms, so a bandit serialises to a plain Hash and back for persistence.

bandit = SkiftetStatistical::Bandit.new(
arms: %w[facebook whatsapp x],
policy: SkiftetStatistical::Policies::ThompsonSampling.new,
)
choice = bandit.select          # => "whatsapp"
bandit.record("whatsapp", 1)    # a conversion
bandit.best_arm                 # => highest empirical mean

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(arms: [], policy: nil) ⇒ Bandit

Returns a new instance of Bandit.



18
19
20
21
22
 
# File 'lib/skiftet_statistical/bandit.rb', line 18

def initialize(arms: [], policy: nil)
  @arms = {}
  Array(arms).each { |a| add_arm(a) }
  @policy = policy || Policies::ThompsonSampling.new
end

Instance Attribute Details

#policy ⇒ Object (readonly)

Returns the value of attribute policy.



16
17
18
 
# File 'lib/skiftet_statistical/bandit.rb', line 16

def policy
  @policy
end

Class Method Details

.from_h(hash, policy: nil) ⇒ Object

Rebuild a bandit's ARM STATE from a hash produced by #to_h. The policy is not reconstructed from its serialised config (policies carry an RNG); pass the policy instance you want to run with.



77
78
79
80
81
 
# File 'lib/skiftet_statistical/bandit.rb', line 77

def self.from_h(hash, policy: nil)
  h = hash.transform_keys(&:to_sym)
  arms = Array(h[:arms]).map { |ah| Arm.from_h(ah) }
  new(arms: arms, policy: policy)
end

Instance Method Details

#add_arm(arm) ⇒ Object

Add an arm by name (String/Symbol) or an existing Arm. Idempotent — an already-known name is left untouched. Returns the arm.



26
27
28
29
 
# File 'lib/skiftet_statistical/bandit.rb', line 26

def add_arm(arm)
  a = arm.is_a?(Arm) ? arm : Arm.new(arm)
  @arms[a.name] ||= a
end

#arm(name) ⇒ Object 



31
32
33
 
# File 'lib/skiftet_statistical/bandit.rb', line 31

def arm(name)
  @arms.fetch(name) { raise Error, "unknown arm: #{name.inspect}" }
end

#arm_names ⇒ Object 



39
40
41
 
# File 'lib/skiftet_statistical/bandit.rb', line 39

def arm_names
  @arms.keys
end

#arms ⇒ Object 



35
36
37
 
# File 'lib/skiftet_statistical/bandit.rb', line 35

def arms
  @arms.values
end

#best_arm ⇒ Object

The arm with the highest empirical mean — the current exploitation choice.



57
58
59
60
61
 
# File 'lib/skiftet_statistical/bandit.rb', line 57

def best_arm
  return nil if @arms.empty?

  @arms.values.max_by(&:mean)&.name
end

#record(name, reward) ⇒ Object

Record an observed reward for the named arm. Returns self for chaining.



51
52
53
54
 
# File 'lib/skiftet_statistical/bandit.rb', line 51

def record(name, reward)
  arm(name).update(reward)
  self
end

#select ⇒ Object

Choose an arm to play. Returns the arm's name.

Raises:



44
45
46
47
48
 
# File 'lib/skiftet_statistical/bandit.rb', line 44

def select
  raise Error, "bandit has no arms" if @arms.empty?

  @policy.choose(@arms.values).name
end

#stats ⇒ Object

Per-arm summary: { name => { pulls:, mean:, reward_sum: } }.



64
65
66
67
68
 
# File 'lib/skiftet_statistical/bandit.rb', line 64

def stats
  @arms.transform_values do |a|
    { pulls: a.pulls, mean: a.mean, reward_sum: a.reward_sum }
  end
end

#to_h ⇒ Object 



70
71
72
 
# File 'lib/skiftet_statistical/bandit.rb', line 70

def to_h
  { arms: @arms.values.map(&:to_h), policy: @policy.to_h }
end