Class: Anthropic::BetaRefusalFallbackMiddleware

Inherits:
Object
  • Object
show all
Includes:
Middleware
Defined in:
lib/anthropic/helpers/refusal_fallback.rb,
sig/anthropic/helpers/refusal_fallback.rbs

Overview

Middleware that retries refused /v1/messages requests down a fallback chain.

Non-streaming: when a response comes back with stop_reason: "refusal", the request is retried with each entry of fallbacks applied as a patch against the original params (a set field overrides, an explicit nil unsets, an absent field keeps the original value) — passing along the refusal's fallback_credit_token — until a model accepts or the chain is exhausted. A message served by a fallback carries a fallback content block prepended at each model boundary; an exhausted chain surfaces the final refusal verbatim.

Streaming: when the stream ends in stop_reason: "refusal", a second request is issued to the fallback model — carrying the refused model's partial output as a trailing assistant prefill when the refusal grants one (fallback_has_prefill_claim), plus the refusal's fallback_credit_token — and the fallback's events are spliced onto the still-open stream, so the client sees one continuous message: a fallback content block at each model boundary, monotonic block indices, and per-hop usage.iterations on the final message_delta. Only model is honored from each entry on this path: the credit token is redeemable only against the refused request's body, so the other per-entry overrides would be rejected.

Credit tokens are sent in the object form with mode: :best_effort, so a failing redemption never rejects the retry — it proceeds at normal price and the outcome is reported on usage.fallback_credit. The fallback-credit beta this requires is sent by default on every request the middleware handles; the betas: option controls this.

To keep later requests on the model that accepted, pass a BetaFallbackState via the fallback_state request option; requests sharing that state start directly at the pinned fallback.

Examples:

client = Anthropic::Client.new(
  middleware: [Anthropic::BetaRefusalFallbackMiddleware.new([{model: "claude-opus-4-8"}])]
)

state = Anthropic::BetaFallbackState.new
message = client.beta.messages.create(**params, request_options: {fallback_state: state})

Defined Under Namespace

Classes: BlockTracker

Constant Summary collapse

DEFAULT_BETAS =

Betas sent by default; override with the betas: option.

Returns:

  • (::Array[String]) 
["fallback-credit-2026-07-01"].freeze
Anthropic =

Returns:

  • (:APIRequest request,) 

      
    

  







  
    

      Instance Method Summary
      collapse
    


    
    
  


  
  
  
  
  
  
  
  
  
  
Methods included from Middleware

  
build_chain



  
Constructor Details

  
    

  

  
    #initialize(fallbacks, betas: DEFAULT_BETAS)  ⇒ BetaRefusalFallbackMiddleware 
  

  

  


  

    
Returns a new instance of BetaRefusalFallbackMiddleware.


  





  
Parameters:


    
  
    
  • 
      
        fallbacks
      
      
        (Array<Hash, Anthropic::Models::Beta::BetaFallbackParam>)
      
      
      
        —
        
    the fallback chain, tried in order. Each entry must carry model:; on
the non-streaming path the remaining keys (max_tokens:, thinking:,
…) patch the original request body for that hop — a value overrides
the param, an explicit nil unsets it, an absent key keeps the
original value.
    
      
    
    • 
  
    
  • 
      
        betas
      
      
        (Array<String>)
      
      
        (defaults to: DEFAULT_BETAS)
      
      
        —
        
    betas added to the anthropic-beta header
of every /v1/messages request this middleware handles. Defaults to
["fallback-credit-2026-07-01"]; pass [] to send none.
    
      
    
    • 
  






  

    
      


76
77
78
79
80
81
82
83
84
85
86

    		
    
      
# File 'lib/anthropic/helpers/refusal_fallback.rb', line 76

def initialize(fallbacks, betas: DEFAULT_BETAS)
  # Entry keys are symbolized because they patch the symbol-keyed request
  # body; a String key would never match, so its override/unset would
  # silently miss.
  @fallbacks = fallbacks.map do |entry|
    entry = entry.to_h if entry.respond_to?(:to_h)
    entry.is_a?(Hash) ? entry.transform_keys(&:to_sym) : entry
  end.freeze
  @betas = betas.freeze
  @warned_missing_state = false
end

    		
  





  




  

    
Instance Method Details


    
      

  

  
    #call(req, nxt)  ⇒ Anthropic::APIResponse 
  

  

  


  

    

  





  
Parameters:




Returns:





  

    
      


91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150

    		
    
      
# File 'lib/anthropic/helpers/refusal_fallback.rb', line 91

def call(req, nxt)
  return nxt.call(req) unless applies?(req)

  if req.body.key?(:fallbacks)
    raise Anthropic::Errors::Error,
          "Sending the `fallbacks:` request param is not supported when using " \
          "`BetaRefusalFallbackMiddleware`. Either remove the middleware and send " \
          "`fallbacks:` with the `server-side-fallback-2026-07-01` beta header to " \
          "let the API handle refusal fallbacks, or omit `fallbacks:` to let the " \
          "middleware handle them on the client side."
  end

  req = with_middleware_headers(req)
  body = strip_fallback_blocks(req.body)
  state = req.options[:fallback_state]

  start_index = state&.index || -1
  unless start_index.is_a?(Integer) && start_index >= -1 && start_index < @fallbacks.length
    raise Anthropic::Errors::Error,
          "fallback_state.index #{start_index} is out of bounds for a chain of " \
          "#{@fallbacks.length} fallback(s); was the state shared with a different middleware?"
  end

  pin = lambda do |index|
    if state
      state.index = index
    elsif !@warned_missing_state
      @warned_missing_state = true
      warn(
        "anthropic-sdk: BetaRefusalFallbackMiddleware fell back without a `fallback_state` " \
        "request option; follow-up requests will retry models that already refused. Pass a " \
        "shared `request_options: {fallback_state: Anthropic::BetaFallbackState.new}` to pin " \
        "them to the accepted model."
      )
    end
  end

  streaming = req.streaming?
  initial_body =
    if start_index == -1
      body
    elsif streaming
      # Only `model` is honored on the streaming path so the credit token
      # — redeemable only against the refused request's body — stays valid
      # for the spliced hops.
      body.merge(model: @fallbacks.fetch(start_index).fetch(:model))
    else
      apply_fallback(body, @fallbacks.fetch(start_index))
    end
  res = nxt.call(req.with(body: initial_body))
  return res unless res.status < 300

  if streaming
    first_hop = start_index + 1
    return res unless first_hop < @fallbacks.length
    return splice_fallback_stream(req, res, nxt, body, first_hop, pin)
  end

  handle_non_streaming(req, res, nxt, body, start_index, pin)
end