Class: RubyLLM::Providers::OpenAIResponses::Batch

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby_llm/providers/openai_responses/batch.rb

Overview

High-level interface for OpenAI’s Batch API. Hides JSONL serialization, file upload, polling, and result parsing behind a clean Ruby API that mirrors RubyLLM::Chat.

Examples:

batch = RubyLLM.batch(model: 'gpt-4o', provider: :openai_responses)
batch.add("What is Ruby?")
batch.add("What is Python?", instructions: "Be brief")
batch.create!
batch.wait! { |b| puts "#{b.completed_count}/#{b.total_count}" }
batch.results  # => { "request_0" => Message, ... }

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model: nil, provider: :openai_responses, id: nil) ⇒ Batch

Returns a new instance of Batch.

Parameters:

  • model (String) (defaults to: nil) —

    Model ID (e.g. ‘gpt-4o’)

  • provider (Symbol, RubyLLM::Providers::OpenAIResponses) (defaults to: :openai_responses) —

    Provider slug or instance

  • id (String, nil) (defaults to: nil) —

    Existing batch ID to resume



25
26
27
28
29
30
31
32
33
34
35
36
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 25

def initialize(model: nil, provider: :openai_responses, id: nil)
  @model = model
  @provider = resolve_provider(provider)
  @requests = []
  @request_counter = 0
  @data = {}

  return unless id

  @id = id
  refresh!
end

Instance Attribute Details

#id ⇒ Object (readonly)

Returns the value of attribute id.



20
21
22
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 20

def id
  @id
end

#requests ⇒ Object (readonly)

Returns the value of attribute requests.



20
21
22
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 20

def requests
  @requests
end

Instance Method Details

#add(input, id: nil, instructions: nil, temperature: nil, tools: nil, **extra) ⇒ self

Queue a request for inclusion in the batch.

Parameters:

  • input (String, Array) —

    User message or Responses API input array

  • id (String, nil) (defaults to: nil) —

    Custom ID for this request (auto-generated if omitted)

  • instructions (String, nil) (defaults to: nil) —

    System/developer instructions

  • temperature (Float, nil) (defaults to: nil) —

    Sampling temperature

  • tools (Array, nil) (defaults to: nil) —

    Tools configuration

Returns:

  • (self) 


45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 45

def add(input, id: nil, instructions: nil, temperature: nil, tools: nil, **extra) # rubocop:disable Metrics/ParameterLists
  custom_id = id || "request_#{@request_counter}"
  @request_counter += 1

  body = { model: @model, input: Batches.normalize_input(input) }
  body[:instructions] = instructions if instructions
  body[:temperature] = temperature if temperature
  body[:tools] = tools if tools
  body.merge!(extra) unless extra.empty?

  @requests << { custom_id: custom_id, body: body }
  self
end

#cancel! ⇒ self

Cancel the batch.

Returns:

  • (self)

Raises:

  • (Error) 


183
184
185
186
187
188
189
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 183

def cancel!
  raise Error.new(nil, 'Batch not yet created') unless @id

  response = @provider.instance_variable_get(:@connection).post(Batches.cancel_batch_url(@id), {})
  @data = response.body
  self
end

#cancelled? ⇒ Boolean

Returns:

  • (Boolean) 


133
134
135
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 133

def cancelled?
  status == Batches::CANCELLED
end

#completed? ⇒ Boolean

Returns:

  • (Boolean) 


113
114
115
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 113

def completed?
  status == Batches::COMPLETED
end

#completed_count ⇒ Integer?

Returns Number of completed requests.

Returns:

  • (Integer, nil) —

    Number of completed requests



98
99
100
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 98

def completed_count
  @data.dig('request_counts', 'completed')
end

#create!(metadata: nil) ⇒ self

Build JSONL, upload the file, and create the batch.

Parameters:

  • metadata (Hash, nil) (defaults to: nil) —

    Optional metadata for the batch

Returns:

  • (self)

Raises:

  • (Error) 


62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 62

def create!(metadata: nil)
  raise Error.new(nil, 'No requests added') if @requests.empty?
  raise Error.new(nil, 'Batch already created') if @id

  jsonl = Batches.build_jsonl(@requests)
  file_id = upload_file(jsonl)

  payload = {
    input_file_id: file_id,
    endpoint: '/v1/responses',
    completion_window: '24h'
  }
  payload[:metadata] = metadata if metadata

  response = @provider.instance_variable_get(:@connection).post(Batches.batches_url, payload)
  @data = response.body
  @id = @data['id']
  self
end

#errors ⇒ Array<Hash>

Download and parse the error file.

Returns:

  • (Array<Hash>) —

    Error entries



173
174
175
176
177
178
179
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 173

def errors
  error_file_id = @data['error_file_id']
  return [] unless error_file_id

  jsonl = fetch_file_content(error_file_id)
  Batches.parse_errors(jsonl)
end

#expired? ⇒ Boolean

Returns:

  • (Boolean) 


128
129
130
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 128

def expired?
  status == Batches::EXPIRED
end

#failed? ⇒ Boolean

Returns:

  • (Boolean) 


123
124
125
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 123

def failed?
  status == Batches::FAILED
end

#failed_count ⇒ Integer?

Returns Number of failed requests.

Returns:

  • (Integer, nil) —

    Number of failed requests



108
109
110
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 108

def failed_count
  @data.dig('request_counts', 'failed')
end

#in_progress? ⇒ Boolean

Returns:

  • (Boolean) 


118
119
120
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 118

def in_progress?
  Batches.pending?(status)
end

#refresh! ⇒ self

Fetch the latest batch status from the API.

Returns:

  • (self)

Raises:

  • (Error) 


84
85
86
87
88
89
90
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 84

def refresh!
  raise Error.new(nil, 'Batch not yet created') unless @id

  response = @provider.instance_variable_get(:@connection).get(Batches.batch_url(@id))
  @data = response.body
  self
end

#results ⇒ Hash<String, Message>

Download and parse the output file into a Hash of Messages.

Returns:

  • (Hash<String, Message>) —

    Results keyed by custom_id

Raises:

  • (Error) 


163
164
165
166
167
168
169
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 163

def results
  output_file_id = @data['output_file_id']
  raise Error.new(nil, 'No output file available yet') unless output_file_id

  jsonl = fetch_file_content(output_file_id)
  Batches.parse_results_to_messages(jsonl)
end

#status ⇒ String?

Returns Batch status.

Returns:

  • (String, nil) —

    Batch status



93
94
95
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 93

def status
  @data['status']
end

#total_count ⇒ Integer?

Returns Total number of requests.

Returns:

  • (Integer, nil) —

    Total number of requests



103
104
105
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 103

def total_count
  @data.dig('request_counts', 'total')
end

#wait!(interval: 30, timeout: nil) {|Batch| ... } ⇒ self

Block until the batch reaches a terminal status.

Parameters:

  • interval (Numeric) (defaults to: 30) —

    Seconds between polls (default: 30)

  • timeout (Numeric, nil) (defaults to: nil) —

    Maximum seconds to wait

Yields:

  • (Batch) —

    Called after each poll

Returns:

  • (self) 


142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
 
# File 'lib/ruby_llm/providers/openai_responses/batch.rb', line 142

def wait!(interval: 30, timeout: nil)
  start_time = Time.now

  loop do
    refresh!
    yield self if block_given?

    break if Batches.terminal?(status)

    if timeout && (Time.now - start_time) > timeout
      raise Error.new(nil, "Batch polling timeout after #{timeout} seconds")
    end

    sleep interval
  end

  self
end