Module: Cohere::Transcribe::GGUF

Defined in:

lib/cohere/transcribe/gguf_writer.rb

Overview

Small GGUF v3 writer tailored to streaming model conversion. Tensor data is supplied by callbacks, so the writer never retains model weights.

Defined Under Namespace

Classes: Error, Metadata, Tensor, Writer

Constant Summary

MAGIC =

"GGUF".b.freeze

VERSION =

3

DEFAULT_ALIGNMENT =

32

VALUE_TYPES =

{
  uint8: 0,
  int8: 1,
  uint16: 2,
  int16: 3,
  uint32: 4,
  int32: 5,
  float32: 6,
  bool: 7,
  string: 8,
  array: 9,
  uint64: 10,
  int64: 11,
  float64: 12
}.freeze

TENSOR_TYPES =

{
  f32: 0,
  f16: 1,
  bf16: 30
}.freeze

TENSOR_WIDTHS =

{
  f32: 4,
  f16: 2,
  bf16: 2
}.freeze