Class: LlmLogs::Batch::Submitter

Inherits:
Object
  • Object
show all
Defined in:
app/models/llm_logs/batch/submitter.rb

Overview

Groups pending BatchRequests of one purpose+model into a single OpenAI batch via ruby_llm-responses_api. To prevent two concurrent FlushJobs from double-submitting the same requests, it first CLAIMS the pending rows in a ‘FOR UPDATE SKIP LOCKED` transaction (assigning them to a placeholder Batch with no openai_batch_id, which flips them out of the `pending` scope and which PollJob ignores). It then submits to OpenAI and records the batch id. If submission fails, the claim is released (requests return to `pending`) and the placeholder batch is dropped, so the work retries next flush.

Instance Method Summary collapse

Constructor Details

#initialize(purpose:, model:, metadata: {}) ⇒ Submitter

Returns a new instance of Submitter.



11
12
13
14
15
 
# File 'app/models/llm_logs/batch/submitter.rb', line 11

def initialize(purpose:, model:, metadata: {})
  @purpose = purpose
  @model = model
  @metadata = metadata
end

Instance Method Details

#call ⇒ Object 



17
18
19
20
21
22
23
 
# File 'app/models/llm_logs/batch/submitter.rb', line 17

def call
  batch = claim_batch
  return nil if batch.nil?

  submit(batch)
  batch
end