Module: RubyLLM::BedrockInvoke::ToolSearch

Defined in:
lib/ruby_llm/bedrock_invoke/tool_search.rb

Overview

Payload shaping for Anthropic's tool search tool on Bedrock InvokeModel.

Bedrock exposes tool search only through InvokeModel (not Converse), as a beta: the request body needs anthropic_beta: ["tool-search-tool-2025-10-19"] and the unversioned tool_search_tool_regex tool type (the first-party API uses versioned types and no beta flag; no BM25 variant exists on Bedrock). Tools marked deferred still send their full definition every request with defer_loading: true — they are excluded from the cached prompt prefix until the model discovers them via the search tool.

Constant Summary collapse

BETA_FLAG = 
'tool-search-tool-2025-10-19'
SEARCH_TOOL_TYPE = 
'tool_search_tool_regex'
SEARCH_TOOL_NAME = 
'tool_search_tool_regex'
SERVER_BLOCK_TYPES = 
%w[server_tool_use tool_search_tool_result].freeze

Class Method Summary collapse

Class Method Details

.apply!(payload, tools) ⇒ Object

Mutates a rendered Anthropic Messages payload: flags deferred tool definitions, prepends the search tool, and opts into the beta. Tools may be marked via the Deferred module or by carrying defer_loading in their own provider params. No-op when nothing is deferred.



36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 36

def apply!(payload, tools)
  names = deferred_names(tools)
  definitions = payload[:tools]

  if definitions.nil?
    return payload if names.empty?

    raise RubyLLM::BedrockInvoke::Error, 'Deferred tools present but payload has no tools'
  end

  definitions.each do |definition|
    name = definition_value(definition, :name).to_s
    definition[:defer_loading] = true if names.include?(name)
  end

  return payload unless definitions.any? { |d| deferred_definition?(d) }

  inject_search_tool!(definitions)
  mark_cache_breakpoint!(definitions)
  payload[:anthropic_beta] = Array(payload[:anthropic_beta]) | [BETA_FLAG]
  payload
end

.block_type(block) ⇒ Object 



97
98
99
100
101
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 97

def block_type(block)
  return nil unless block.is_a?(Hash)

  (block[:type] || block['type']).to_s
end

.contains_server_blocks?(blocks) ⇒ Boolean

Returns:

  • (Boolean) 


93
94
95
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 93

def contains_server_blocks?(blocks)
  Array(blocks).any? { |block| server_block?(block) }
end

.deferred?(tool) ⇒ Boolean

Returns:

  • (Boolean) 


22
23
24
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 22

def deferred?(tool)
  tool.respond_to?(:defer_loading?) && tool.defer_loading?
end

.deferred_definition?(definition) ⇒ Boolean

Returns:

  • (Boolean) 


103
104
105
106
107
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 103

def deferred_definition?(definition)
  return false unless definition.is_a?(Hash)

  !!definition_value(definition, :defer_loading)
end

.deferred_names(tools) ⇒ Object

tools is RubyLLM's name => tool hash; returns name strings.



27
28
29
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 27

def deferred_names(tools)
  tools.values.select { |tool| deferred?(tool) }.map { |tool| tool.name.to_s }
end

.definition_value(definition, key) ⇒ Object 



115
116
117
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 115

def definition_value(definition, key)
  definition[key] || definition[key.to_s]
end

.ensure_beta!(payload) ⇒ Object

Both generations deep-merge user params (with_params / with_provider_options) over the rendered payload AFTER apply! runs, and deep_merge REPLACES arrays — silently dropping the beta flag. Called on the final merged payload right before the request goes out.



79
80
81
82
83
84
85
86
87
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 79

def ensure_beta!(payload)
  definitions = payload[:tools] || payload['tools']
  return payload unless definitions.is_a?(Array)
  return payload unless definitions.any? { |d| deferred_definition?(d) || search_tool_definition?(d) }

  betas = Array(payload.delete('anthropic_beta')) | Array(payload[:anthropic_beta]) | [BETA_FLAG]
  payload[:anthropic_beta] = betas
  payload
end

.inject_search_tool!(definitions) ⇒ Object 



119
120
121
122
123
124
125
126
127
128
129
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 119

def inject_search_tool!(definitions)
  return if definitions.any? { |d| search_tool_definition?(d) }

  collision = definitions.find { |d| definition_value(d, :name).to_s == SEARCH_TOOL_NAME }
  if collision
    raise RubyLLM::BedrockInvoke::Error,
          "A tool named '#{SEARCH_TOOL_NAME}' collides with the injected tool search tool. Rename it."
  end

  definitions.unshift(type: SEARCH_TOOL_TYPE, name: SEARCH_TOOL_NAME)
end

.mark_cache_breakpoint!(definitions) ⇒ Object

Cache the eager (non-deferred) tool-definitions prefix. Deferred tools can't carry cache_control (the API returns 400) and are excluded from the cached prefix anyway, so the breakpoint goes on the last NON-deferred tool. This preserves prompt caching across turns (the discovered tools load inline as tool_reference blocks, never touching this prefix) and, crucially, gives the request the >=1 cache_control marker that turns on the API's automatic caching of server tool-search results.



66
67
68
69
70
71
72
73
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 66

def mark_cache_breakpoint!(definitions)
  return if definitions.any? { |d| d.is_a?(Hash) && definition_value(d, :cache_control) }

  last_eager = definitions.reverse.find { |d| d.is_a?(Hash) && !deferred_definition?(d) }
  return unless last_eager

  last_eager[:cache_control] = { type: 'ephemeral' }
end

.search_tool_definition?(definition) ⇒ Boolean

Returns:

  • (Boolean) 


109
110
111
112
113
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 109

def search_tool_definition?(definition)
  return false unless definition.is_a?(Hash)

  definition_value(definition, :type) == SEARCH_TOOL_TYPE
end

.server_block?(block) ⇒ Boolean

Returns:

  • (Boolean) 


89
90
91
 
# File 'lib/ruby_llm/bedrock_invoke/tool_search.rb', line 89

def server_block?(block)
  SERVER_BLOCK_TYPES.include?(block_type(block))
end