Class: Cohere::Transcribe::ASR::BatchExecutor

Inherits:
Object
  • Object
show all
Defined in:
lib/cohere/transcribe/asr/batching.rb

Overview

Generic recursive recovery around a native batch operation. Results remain in input order, data-local failures are isolated, fatal failures open a persistent circuit, and learned caps are honored by siblings that have not yet been attempted.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(controller, duration: nil, memory: nil, operation_metrics: nil, telemetry: BatchTelemetry.new) ⇒ BatchExecutor

Returns a new instance of BatchExecutor.



279
280
281
282
283
284
285
 
# File 'lib/cohere/transcribe/asr/batching.rb', line 279

def initialize(controller, duration: nil, memory: nil, operation_metrics: nil, telemetry: BatchTelemetry.new)
  @controller = controller
  @duration = duration || method(:default_duration)
  @memory = memory
  @operation_metrics = operation_metrics
  @telemetry = telemetry
end

Instance Attribute Details

#telemetry ⇒ Object (readonly)

Returns the value of attribute telemetry.



277
278
279
 
# File 'lib/cohere/transcribe/asr/batching.rb', line 277

def telemetry
  @telemetry
end

Instance Method Details

#execute(rows, max_new_tokens:, base_max_new_tokens:, retry_cap: nil, &operation) ⇒ Object

Raises:

  • (ArgumentError) 


287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
 
# File 'lib/cohere/transcribe/asr/batching.rb', line 287

def execute(rows, max_new_tokens:, base_max_new_tokens:, retry_cap: nil, &operation)
  raise ArgumentError, "a native batch operation block is required" unless operation

  source = rows.to_a
  values = Array.new(source.length)
  errors = Array.new(source.length)
  indexed = source.each_with_index.map { |row, index| [index, row] }
  high_token_retry = max_new_tokens > base_max_new_tokens
  retry_cap ||= RetryBatchCap.new([source.length, 1].max) if high_token_retry
  process(
    indexed,
    values,
    errors,
    max_new_tokens: max_new_tokens,
    high_token_retry: high_token_retry,
    retry_cap: retry_cap,
    operation: operation
  )
  @telemetry.finalize(@controller)
  BatchExecutionResult.new(values: values.freeze, errors: errors.freeze, telemetry: @telemetry)
end