Module: RubyLLM::Providers::OpenAIResponses::BuiltInTools

Defined in:
lib/ruby_llm/providers/openai_responses/built_in_tools.rb

Overview

Built-in tools support for the OpenAI Responses API. Provides configuration helpers and result parsing for:

  • Web Search

  • File Search

  • Code Interpreter

  • Image Generation

  • MCP (Model Context Protocol)

Constant Summary collapse

CALL_ARGUMENT_EXTRACTORS =

Server-executed built-in tool output item types and their argument extractors. The key is the output ‘type` from the Responses API; the value is a lambda that pulls the relevant arguments out of that item. To support a new built-in tool, add an entry here.

{
  'web_search_call' => ->(item) { { action: item['action'], query: item.dig('action', 'query') } },
  'file_search_call' => ->(item) { { queries: item['queries'] } },
  'code_interpreter_call' => ->(item) { { code: item['code'], container_id: item['container_id'] } },
  'image_generation_call' => ->(_item) { {} },
  'shell_call' => ->(item) { { action: item['action'], container_id: item['container_id'] } },
  'local_shell_call' => ->(item) { { action: item['action'], container_id: item['container_id'] } },
  'apply_patch_call' => ->(item) { { operation: item['operation'] } },
  'mcp_call' => lambda { |item|
    { name: item['name'], arguments: item['arguments'], server_label: item['server_label'] }
  },
  'computer_call' => ->(item) { { action: item['action'] } }
}.freeze

Class Method Summary collapse

Class Method Details

.apply_patch ⇒ Object

Apply Patch tool configuration Enables the model to create, update, and delete files using structured diffs.



115
116
117
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 115

def apply_patch
  { type: 'apply_patch' }
end

.code_interpreter(container_type: 'auto') ⇒ Object

Code Interpreter tool configuration

Parameters:

  • container_type (String) (defaults to: 'auto') —

    ‘auto’ or specific container type



48
49
50
51
52
53
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 48

def code_interpreter(container_type: 'auto')
  {
    type: 'code_interpreter',
    container: { type: container_type }
  }
end

.computer_use(display_width:, display_height:, environment: 'browser') ⇒ Object

Computer Use tool configuration (preview)

Parameters:

  • display_width (Integer) —

    Display width in pixels

  • display_height (Integer) —

    Display height in pixels

  • environment (String) (defaults to: 'browser') —

    ‘browser’ or ‘mac’ or ‘windows’ or ‘ubuntu’



85
86
87
88
89
90
91
92
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 85

def computer_use(display_width:, display_height:, environment: 'browser')
  {
    type: 'computer_use_preview',
    display_width: display_width,
    display_height: display_height,
    environment: environment
  }
end

.extract_citations(content) ⇒ Array<Hash>

Extract all citations from message content

Parameters:

  • content (Array) —

    Message content array

Returns:

  • (Array<Hash>) —

    All citations/annotations



278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 278

def extract_citations(content)
  return [] unless content.is_a?(Array)

  content
    .select { |c| c['type'] == 'output_text' }
    .flat_map { |c| c['annotations'] || [] }
    .map do |annotation|
      {
        type: annotation['type'],
        text: annotation['text'],
        url: annotation['url'],
        title: annotation['title'],
        start_index: annotation['start_index'],
        end_index: annotation['end_index']
      }.compact
    end
end

.extract_events(output) ⇒ Object

Build a list of result: events from the response output. Used to surface server-side built-in tool activity through the standard on_tool_call / on_tool_result callbacks (issue #1).



217
218
219
220
221
222
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 217

def extract_events(output)
  return [] unless output.is_a?(Array)

  output.select { |item| CALL_ARGUMENT_EXTRACTORS.key?(item['type']) }
        .map { |item| build_event(item) }
end

.file_search(vector_store_ids:, max_num_results: nil, ranking_options: nil) ⇒ Object

File Search tool configuration

Parameters:

  • vector_store_ids (Array<String>) —

    IDs of vector stores to search

  • max_num_results (Integer, nil) (defaults to: nil) —

    Maximum results to return

  • ranking_options (Hash, nil) (defaults to: nil) —

    Ranking configuration



36
37
38
39
40
41
42
43
44
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 36

def file_search(vector_store_ids:, max_num_results: nil, ranking_options: nil)
  tool = {
    type: 'file_search',
    vector_store_ids: Array(vector_store_ids)
  }
  tool[:max_num_results] = max_num_results if max_num_results
  tool[:ranking_options] = ranking_options if ranking_options
  tool
end

.image_generation(partial_images: nil) ⇒ Object

Image Generation tool configuration

Parameters:

  • partial_images (Integer, nil) (defaults to: nil) —

    Number of partial images during streaming



57
58
59
60
61
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 57

def image_generation(partial_images: nil)
  tool = { type: 'image_generation' }
  tool[:partial_images] = partial_images if partial_images
  tool
end

.mcp(server_label:, server_url:, require_approval: 'never', allowed_tools: nil, headers: nil) ⇒ Object

MCP (Model Context Protocol) tool configuration

Parameters:

  • server_label (String) —

    Label for the MCP server

  • server_url (String) —

    URL of the MCP server

  • require_approval (String) (defaults to: 'never') —

    ‘never’, ‘always’, or specific tool patterns

  • allowed_tools (Array<String>, nil) (defaults to: nil) —

    List of allowed tool names

  • headers (Hash, nil) (defaults to: nil) —

    Additional headers for the MCP server



69
70
71
72
73
74
75
76
77
78
79
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 69

def mcp(server_label:, server_url:, require_approval: 'never', allowed_tools: nil, headers: nil)
  tool = {
    type: 'mcp',
    server_label: server_label,
    server_url: server_url,
    require_approval: require_approval
  }
  tool[:allowed_tools] = allowed_tools if allowed_tools
  tool[:headers] = headers if headers
  tool
end

.parse_apply_patch_results(output) ⇒ Array<Hash>

Parse apply_patch call results from output

Parameters:

  • output (Array) —

    Response output array

Returns:

  • (Array<Hash>) —

    Parsed apply_patch call results



183
184
185
186
187
188
189
190
191
192
193
194
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 183

def parse_apply_patch_results(output)
  output
    .select { |item| item['type'] == 'apply_patch_call' }
    .map do |item|
      {
        id: item['id'],
        call_id: item['call_id'],
        status: item['status'],
        operation: item['operation']
      }
    end
end

.parse_code_interpreter_results(output) ⇒ Array<Hash>

Parse code interpreter results from output

Parameters:

  • output (Array) —

    Response output array

Returns:

  • (Array<Hash>) —

    Parsed code interpreter results



152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 152

def parse_code_interpreter_results(output)
  output
    .select { |item| item['type'] == 'code_interpreter_call' }
    .map do |item|
      {
        id: item['id'],
        code: item['code'],
        results: item['results'] || [],
        container_id: item['container_id']
      }
    end
end

.parse_file_search_results(output) ⇒ Array<Hash>

Parse file search results from output

Parameters:

  • output (Array) —

    Response output array

Returns:

  • (Array<Hash>) —

    Parsed file search results



137
138
139
140
141
142
143
144
145
146
147
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 137

def parse_file_search_results(output)
  output
    .select { |item| item['type'] == 'file_search_call' }
    .map do |item|
      {
        id: item['id'],
        status: item['status'],
        results: item['results'] || []
      }
    end
end

.parse_image_generation_results(output) ⇒ Array<Hash>

Parse image generation results from output

Parameters:

  • output (Array) —

    Response output array

Returns:

  • (Array<Hash>) —

    Parsed image generation results



168
169
170
171
172
173
174
175
176
177
178
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 168

def parse_image_generation_results(output)
  output
    .select { |item| item['type'] == 'image_generation_call' }
    .map do |item|
      {
        id: item['id'],
        status: item['status'],
        result: item['result']
      }
    end
end

.parse_shell_call_results(output) ⇒ Array<Hash>

Parse shell call results from output

Parameters:

  • output (Array) —

    Response output array

Returns:

  • (Array<Hash>) —

    Parsed shell call results



261
262
263
264
265
266
267
268
269
270
271
272
273
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 261

def parse_shell_call_results(output)
  output
    .select { |item| item['type'] == 'shell_call' }
    .map do |item|
      {
        id: item['id'],
        call_id: item['call_id'],
        status: item['status'],
        action: item['action'],
        container_id: item['container_id']
      }
    end
end

.parse_web_search_results(output) ⇒ Array<Hash>

Parse web search results from output

Parameters:

  • output (Array) —

    Response output array

Returns:

  • (Array<Hash>) —

    Parsed search results with citations



122
123
124
125
126
127
128
129
130
131
132
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 122

def parse_web_search_results(output)
  output
    .select { |item| item['type'] == 'web_search_call' }
    .map do |item|
      {
        id: item['id'],
        status: item['status'],
        results: parse_citations(item)
      }
    end
end

.shell(environment_type: 'container_auto', container_id: nil, network_policy: nil, memory_limit: nil) ⇒ Object

Shell tool configuration

Parameters:

  • environment_type (String) (defaults to: 'container_auto') —

    ‘container_auto’, ‘container_reference’, or ‘local’

  • container_id (String, nil) (defaults to: nil) —

    Container ID for ‘container_reference’ type

  • network_policy (Hash, nil) (defaults to: nil) —

    Network policy (e.g. { type: ‘allowlist’, allowed_domains: […] })

  • memory_limit (String, nil) (defaults to: nil) —

    Memory limit: ‘1g’, ‘4g’, ‘16g’, ‘64g’



99
100
101
102
103
104
105
106
107
108
109
110
111
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 99

def shell(environment_type: 'container_auto', container_id: nil,
          network_policy: nil, memory_limit: nil)
  env = if container_id
          { type: 'container_reference', container_id: container_id }
        else
          { type: environment_type }
        end

  env[:network_policy] = network_policy if network_policy
  env[:memory_limit] = memory_limit if memory_limit

  { type: 'shell', environment: env }
end

.web_search(search_context_size: nil, user_location: nil, preview: false) ⇒ Object

Web Search tool configuration

Parameters:

  • search_context_size (String, nil) (defaults to: nil) —

    ‘low’, ‘medium’, or ‘high’

  • user_location (Hash, nil) (defaults to: nil) —

    { type: ‘approximate’, city: ‘…’, country: ‘…’ }

  • preview (Boolean) (defaults to: false) —

    use the legacy preview tool type



20
21
22
23
24
25
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 20

def web_search(search_context_size: nil, user_location: nil, preview: false)
  tool = { type: preview ? 'web_search_preview' : 'web_search' }
  tool[:search_context_size] = search_context_size if search_context_size
  tool[:user_location] = user_location if user_location
  tool
end

.web_search_preview(search_context_size: nil, user_location: nil) ⇒ Object

Legacy Web Search preview tool configuration.



28
29
30
 
# File 'lib/ruby_llm/providers/openai_responses/built_in_tools.rb', line 28

def web_search_preview(search_context_size: nil, user_location: nil)
  web_search(search_context_size: search_context_size, user_location: user_location, preview: true)
end