Module: Cohere::Transcribe::Runtime::Precision

Defined in:
lib/cohere/transcribe/runtime/precision.rb

Overview

Resolves public device/precision requests against the native runtime before any input is decoded. The returned options describe what will actually execute; an unavailable explicit accelerator is never allowed to fall through to ggml's command-line-oriented CPU fallback.

Class Method Summary collapse

Class Method Details

.resolve(options, native_library: nil) ⇒ Object 



15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
 
# File 'lib/cohere/transcribe/runtime/precision.rb', line 15

def resolve(options, native_library: nil)
  library = native_library
  device = if options.device == "cpu"
             "cpu"
           else
             library ||= ASR::NativeLibrary.load
             library.resolve_device(options.device)
           end
  dtype = resolved_dtype(options.dtype, device, library)
  if options.alignment == "word" && options.align_dtype == "fp16" && device != "cuda"
    raise TranscriptionRuntimeError,
          "--align-dtype fp16 is supported only with CUDA"
  end

  vad_engine = options.vad == "silero" ? "onnx" : options.vad_engine
  options.with(device: device, dtype: dtype, vad_engine: vad_engine)
rescue TranscriptionError
  raise
rescue Fiddle::DLError, LoadError, SystemCallError => e
  raise TranscriptionRuntimeError,
        "Cannot resolve the native inference device: #{e.class}: #{e.message}"
end