Class: LlmLogs::Batch::Submitter

Inherits:
Object
  • Object
show all
Defined in:
app/models/llm_logs/batch/submitter.rb

Overview

Groups pending BatchRequests of one purpose+model into a single OpenAI batch via ruby_llm-responses_api. To prevent two concurrent FlushJobs from double-submitting the same requests, it first CLAIMS the pending rows in a FOR UPDATE SKIP LOCKED transaction (assigning them to a placeholder Batch with no openai_batch_id, which flips them out of the pending scope and which PollJob ignores). It then submits to OpenAI and records the batch id. If submission fails, the claim is released (requests return to pending) and the placeholder batch is dropped, so the work retries next flush.

Instance Method Summary collapse

Constructor Details

#initialize(purpose:, model:, metadata: {}) ⇒ Submitter

Returns a new instance of Submitter.



11
12
13
14
15
 
# File 'app/models/llm_logs/batch/submitter.rb', line 11

def initialize(purpose:, model:, metadata: {})
  @purpose = purpose
  @model = model
  @metadata = metadata
end

Instance Method Details

#call ⇒ Object 



17
18
19
20
21
22
23
 
# File 'app/models/llm_logs/batch/submitter.rb', line 17

def call
  batch = claim_batch
  return nil if batch.nil?

  submit(batch)
  batch
end