Module: Solace::Utils::Codecs

Extended by:
Codecs
Included in:
Codecs
Defined in:
lib/solace/utils/codecs.rb

Overview

Module for encoding and decoding data.

The helpers are grouped by category: base64/base58 string encodings, fixed-width little-endian integers, the Solana compact-u16 (shortvec) varint, length-prefixed byte/pubkey collections, and optionals.

Serialization formats referenced below:

  • “Borsh” — the Anchor serialization spec. Borsh encodes ‘bool` as a single 0/1 byte, `Vec<T>`/`bytes` with a u32 little-endian length prefix, and `Option<T>` as a 1-byte discriminant (0 = None, 1 = Some) followed by the value. Methods that produce these layouts say so explicitly.

  • “SmallVec” — a Solana/Anchor-program convention (e.g. Squads) that prefixes a collection with a u8 or u16 length instead of Borsh’s u32. NOT Borsh.

  • “compact-u16” / “shortvec” — Solana’s own variable-length integer used in the transaction wire format. NOT Borsh.

‘extend self` exposes every method both as a module method (Solace::Utils::Codecs.<method>) and as an instance method when the module is included into a composed encoder/decoder.

Since:

  • 0.0.1

Instance Method Summary collapse

Instance Method Details

#base58_to_binary(string) ⇒ String

Decodes a Base58 string into a binary string

Parameters:

  • string (String) —

    The Base58 encoded string

Returns:

  • (String) —

    The decoded binary string

Since:

  • 0.0.1



75
76
77
 
# File 'lib/solace/utils/codecs.rb', line 75

def base58_to_binary(string)
  base58_to_bytes(string).pack('C*')
end

#base58_to_bytes(string) ⇒ Array<Integer>

Decodes a Base58 string into a sequence of bytes

Parameters:

  • string (String) —

    The Base58 encoded string

Returns:

  • (Array<Integer>) —

    The decoded bytes

Since:

  • 0.0.1



91
92
93
 
# File 'lib/solace/utils/codecs.rb', line 91

def base58_to_bytes(string)
  Base58.base58_to_binary(string, :bitcoin).bytes
end

#base64_to_bytestream(base64) ⇒ StringIO

Creates a StringIO from a base64 string.

Parameters:

  • base64 (String) —

    The base64 string to decode

Returns:

  • (StringIO) —

    A StringIO object containing the decoded bytes

Since:

  • 0.0.1



57
58
59
 
# File 'lib/solace/utils/codecs.rb', line 57

def base64_to_bytestream(base64)
  StringIO.new(Base64.decode64(base64))
end

#binary_to_base58(binary) ⇒ String

Encodes a binary string in Base58 format.

Parameters:

  • binary (String) —

    The bytes to encode

Returns:

  • (String) —

    The Base58 encoded string

Since:

  • 0.0.1



67
68
69
 
# File 'lib/solace/utils/codecs.rb', line 67

def binary_to_base58(binary)
  Base58.binary_to_base58(binary, :bitcoin)
end

#bytes_to_base58(bytes) ⇒ String

Encodes a byte array in Base58 format

Parameters:

  • bytes (Array<Integer>) —

    The bytes to encode

Returns:

  • (String) —

    The Base58 encoded string

Since:

  • 0.0.1



83
84
85
 
# File 'lib/solace/utils/codecs.rb', line 83

def bytes_to_base58(bytes)
  binary_to_base58(bytes.pack('C*'))
end

#decode_bytes(stream) ⇒ String

Decodes a Borsh bytes / Vec<u8> field: u32 LE length prefix + raw bytes.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (String) —

    The raw bytes as a binary string.

Since:

  • 0.0.1



298
299
300
 
# File 'lib/solace/utils/codecs.rb', line 298

def decode_bytes(stream)
  stream.read(decode_le_u32(stream))
end

#decode_compact_u16(stream) ⇒ Integer

Decodes a compact-u16 (shortvec) value from an IO-like object.

Reads bytes one at a time, accumulating the result until the MSB is 0.

Parameters:

  • stream (IO, StringIO) —

    The input to read bytes from.

Returns:

  • (Integer, Integer) —

    The decoded value and the number of bytes read.

Since:

  • 0.0.1



264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
 
# File 'lib/solace/utils/codecs.rb', line 264

def decode_compact_u16(stream)
  value      = 0
  shift      = 0
  bytes_read = 0

  loop do
    byte = stream.read(1)
    raise EOFError, 'Unexpected end of input while decoding compact-u16' unless byte

    byte        = byte.ord
    value      |= (byte & 0x7F) << shift
    bytes_read += 1
    break if byte.nobits?(0x80)

    shift += 7
  end

  [value, bytes_read]
end

#decode_le_i64(stream) ⇒ Integer

Decodes an i64 from 8 little-endian bytes (two’s complement).

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (Integer) —

    Value in range -2**63..2**63-1.

Since:

  • 0.0.1



203
204
205
 
# File 'lib/solace/utils/codecs.rb', line 203

def decode_le_i64(stream)
  stream.read(8).unpack1('q<')
end

#decode_le_u128(stream) ⇒ Integer

Decodes a u128 from 16 little-endian bytes (two u64 words, low word first).

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (Integer) —

    Value in range 0..2**128-1.

Since:

  • 0.0.1



186
187
188
189
 
# File 'lib/solace/utils/codecs.rb', line 186

def decode_le_u128(stream)
  lo, hi = stream.read(16).unpack('Q<Q<')
  lo + (hi << 64)
end

#decode_le_u16(stream) ⇒ Integer

Decodes a u16 from 2 little-endian bytes.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (Integer) —

    Value in range 0..65535.

Since:

  • 0.0.1



138
139
140
 
# File 'lib/solace/utils/codecs.rb', line 138

def decode_le_u16(stream)
  stream.read(2).unpack1('S<')
end

#decode_le_u32(stream) ⇒ Integer

Decodes a u32 from 4 little-endian bytes.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (Integer) —

    Value in range 0..4294967295.

Since:

  • 0.0.1



154
155
156
 
# File 'lib/solace/utils/codecs.rb', line 154

def decode_le_u32(stream)
  stream.read(4).unpack1('L<')
end

#decode_le_u64(stream) ⇒ Integer

Decodes a little-endian u64 value from a sequence of bytes

Parameters:

  • stream (IO, StringIO) —

    The input to read bytes from.

Returns:

  • (Integer) —

    The decoded u64 value

Since:

  • 0.0.1



170
171
172
 
# File 'lib/solace/utils/codecs.rb', line 170

def decode_le_u64(stream)
  stream.read(8).unpack1('Q<')
end

#decode_option_i64(stream) ⇒ Integer?

Decodes a Borsh Option<i64>: None → nil, Some(i64) → integer.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (Integer, nil)

Since:

  • 0.0.1



406
407
408
409
410
 
# File 'lib/solace/utils/codecs.rb', line 406

def decode_option_i64(stream)
  return nil if decode_u8(stream).zero?

  decode_le_i64(stream)
end

#decode_option_pubkey(stream) ⇒ String?

Decodes a Borsh Option<publicKey>: None → nil, Some(key) → base58 pubkey.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (String, nil) —

    Base58 public key or nil.

Since:

  • 0.0.1



386
387
388
389
390
 
# File 'lib/solace/utils/codecs.rb', line 386

def decode_option_pubkey(stream)
  return nil if decode_u8(stream).zero?

  decode_pubkey(stream)
end

#decode_pubkey(stream) ⇒ String

Decodes a public key from 32 bytes.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (String) —

    Base58 public key.

Since:

  • 0.0.1



336
337
338
 
# File 'lib/solace/utils/codecs.rb', line 336

def decode_pubkey(stream)
  bytes_to_base58(stream.read(32).bytes)
end

#decode_u8(stream) ⇒ Integer

Decodes a u8 from 1 byte.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (Integer) —

    Value in range 0..255.

Since:

  • 0.0.1



122
123
124
 
# File 'lib/solace/utils/codecs.rb', line 122

def decode_u8(stream)
  stream.read(1).unpack1('C')
end

#decode_vec_pubkeys(stream) ⇒ Array<String>

Decodes a Borsh Vec<publicKey>: u32 LE count prefix followed by each 32-byte pubkey.

Parameters:

  • stream (IO, StringIO) —

    The stream to read from.

Returns:

  • (Array<String>) —

    Base58 public keys.

Since:

  • 0.0.1



355
356
357
 
# File 'lib/solace/utils/codecs.rb', line 355

def decode_vec_pubkeys(stream)
  Array.new(decode_le_u32(stream)) { decode_pubkey(stream) }
end

#encode_bool(bool) ⇒ Array<Integer>

Encodes a Borsh bool as a single byte: false → 0, true → 1.

Parameters:

  • bool (Boolean) —

    The value to encode.

Returns:

  • (Array<Integer>) —

    A single-element byte array.

Since:

  • 0.0.1



213
214
215
 
# File 'lib/solace/utils/codecs.rb', line 213

def encode_bool(bool)
  [bool ? 1 : 0]
end

#encode_bytes(bytes) ⇒ Array<Integer>

Encodes a Borsh bytes field / Vec<u8>: u32 LE length prefix + raw bytes.

Parameters:

  • bytes (Array<Integer>) —

    The raw bytes.

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



290
291
292
 
# File 'lib/solace/utils/codecs.rb', line 290

def encode_bytes(bytes)
  encode_le_u32(bytes.length).bytes + bytes
end

#encode_compact_u16(u16) ⇒ String

Encodes an integer as a compact-u16 (shortvec) varint. This is Solana’s transaction wire-format length prefix, NOT Borsh.

Parameters:

  • u16 (Integer) —

    The compact-u16 value to encode

Returns:

  • (String) —

    The compactly encoded compact-u16 value

Since:

  • 0.0.1



224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
 
# File 'lib/solace/utils/codecs.rb', line 224

def encode_compact_u16(u16)
  out = []

  loop do
    # In general, n >> 7 shifts the bits of n to the right by
    # 7 positions, effectively dividing n by 128 and discarding
    # the remainder (integer division). This is commonly used in
    # encoding schemes to process one "byte" (7 bits) at a time.
    if (u16 >> 7).zero?
      out << u16
      break
    end
    # The expression out << ((n & 0x7F) | 0x80) is used in variable-length
    # integer encoding, such as the compact-u16 encoding.
    #
    # n & 0x7F:
    #   - 0x7F is 127 in decimal, or 0111 1111 in binary.
    #   - n & 0x7F masks out all but the lowest 7 bits of n. This extracts the least significant 7 bits of n.
    #
    # (n & 0x7F) | 0x80:
    #   - 0x80 is 128 in decimal, or 1000 0000 in binary.
    #   - | (bitwise OR) sets the highest bit (the 8th bit) to 1.
    #   - This is a signal that there are more bytes to come in the encoding (i.e., the value hasn't been fully
    #     encoded yet).
    #
    # out << ...:
    #   - This appends the resulting byte to the out array.
    out << ((u16 & 0x7F) | 0x80)
    u16 >>= 7
  end

  out.pack('C*')
end

#encode_le_i64(i64) ⇒ String

Encodes an i64 as 8 little-endian bytes (two’s complement).

Parameters:

  • i64 (Integer) —

    Value in range -2**63..2**63-1.

Returns:

  • (String) —

    8-byte little-endian binary string.

Since:

  • 0.0.1



195
196
197
 
# File 'lib/solace/utils/codecs.rb', line 195

def encode_le_i64(i64)
  [i64].pack('q<')
end

#encode_le_u128(u128) ⇒ String

Encodes a u128 as 16 little-endian bytes (two u64 words, low word first).

Parameters:

  • u128 (Integer) —

    Value in range 0..2**128-1.

Returns:

  • (String) —

    16-byte little-endian binary string.

Since:

  • 0.0.1



178
179
180
 
# File 'lib/solace/utils/codecs.rb', line 178

def encode_le_u128(u128)
  [u128 & 0xFFFFFFFFFFFFFFFF, u128 >> 64].pack('Q<Q<')
end

#encode_le_u16(u16) ⇒ String

Encodes a u16 as 2 little-endian bytes.

Parameters:

  • u16 (Integer) —

    Value in range 0..65535.

Returns:

  • (String) —

    2-byte little-endian binary string.

Since:

  • 0.0.1



130
131
132
 
# File 'lib/solace/utils/codecs.rb', line 130

def encode_le_u16(u16)
  [u16].pack('S<')
end

#encode_le_u32(u32) ⇒ String

Encodes a u32 as 4 little-endian bytes.

Parameters:

  • u32 (Integer) —

    Value in range 0..4294967295.

Returns:

  • (String) —

    4-byte little-endian binary string.

Since:

  • 0.0.1



146
147
148
 
# File 'lib/solace/utils/codecs.rb', line 146

def encode_le_u32(u32)
  [u32].pack('L<')
end

#encode_le_u64(u64) ⇒ String

Encodes a u64 value in little-endian format

Parameters:

  • u64 (Integer) —

    The u64 value to encode

Returns:

  • (String) —

    The little-endian encoded u64 value

Since:

  • 0.0.1



162
163
164
 
# File 'lib/solace/utils/codecs.rb', line 162

def encode_le_u64(u64)
  [u64].pack('Q<') # 64-bit little-endian
end

#encode_option_i64(i64) ⇒ Array<Integer>

Encodes a Borsh Option<i64>: None → [0], Some(i64) → [1] + 8 LE bytes.

Parameters:

  • i64 (Integer, nil)

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



396
397
398
399
400
 
# File 'lib/solace/utils/codecs.rb', line 396

def encode_option_i64(i64)
  return [0] if i64.nil?

  [1] + encode_le_i64(i64).bytes
end

#encode_option_pubkey(pubkey) ⇒ Array<Integer>

Encodes a Borsh Option<publicKey>: None → [0], Some(key) → [1] + 32 bytes.

Parameters:

  • pubkey (#to_s, nil) —

    Base58 public key or nil.

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



376
377
378
379
380
 
# File 'lib/solace/utils/codecs.rb', line 376

def encode_option_pubkey(pubkey)
  return [0] if pubkey.nil?

  [1] + encode_pubkey(pubkey)
end

#encode_option_string(str) ⇒ Array<Integer>

Encodes a Borsh Option<String>: None → [0], Some(str) → [1] + u32 LE length + UTF-8 bytes.

Parameters:

  • str (String, nil)

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



417
418
419
420
421
422
 
# File 'lib/solace/utils/codecs.rb', line 417

def encode_option_string(str)
  return [0] if str.nil?

  bytes = str.encode('UTF-8').bytes
  [1] + encode_le_u32(bytes.length).bytes + bytes
end

#encode_pubkey(pubkey) ⇒ Array<Integer>

Encodes a public key as 32 raw bytes (a Solana primitive, not a Borsh type). Accepts any representation that resolves to a base58 string via #to_s (String, Keypair, PublicKey).

Parameters:

  • pubkey (#to_s) —

    The public key in any representation.

Returns:

  • (Array<Integer>) —

    32 bytes.

Since:

  • 0.0.1



328
329
330
 
# File 'lib/solace/utils/codecs.rb', line 328

def encode_pubkey(pubkey)
  base58_to_bytes(pubkey.to_s)
end

#encode_smallvec_u16_bytes(bytes) ⇒ Array<Integer>

Encodes a SmallVec<u16, u8>: u16 LE length prefix + raw bytes. SmallVec is a Solana/Anchor-program convention, NOT Borsh (which would use a u32 prefix).

Parameters:

  • bytes (Array<Integer>) —

    The raw bytes (max 65535).

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



316
317
318
 
# File 'lib/solace/utils/codecs.rb', line 316

def encode_smallvec_u16_bytes(bytes)
  encode_le_u16(bytes.length).bytes + bytes
end

#encode_smallvec_u8_bytes(bytes) ⇒ Array<Integer>

Encodes a SmallVec<u8, u8>: u8 length prefix + raw bytes. SmallVec is a Solana/Anchor-program convention, NOT Borsh (which would use a u32 prefix).

Parameters:

  • bytes (Array<Integer>) —

    The raw bytes (max 255).

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



307
308
309
 
# File 'lib/solace/utils/codecs.rb', line 307

def encode_smallvec_u8_bytes(bytes)
  [bytes.length] + bytes
end

#encode_smallvec_u8_pubkeys(pubkeys) ⇒ Array<Integer>

Encodes a SmallVec<u8, Pubkey>: u8 count prefix followed by each 32-byte pubkey. SmallVec is a Solana convention (NOT Borsh); used by the transaction message header’s account_keys (distinct from encode_vec_pubkeys, which uses Borsh’s u32 count).

Parameters:

  • pubkeys (Array<#to_s>) —

    The public keys in any representation (max 255).

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



366
367
368
 
# File 'lib/solace/utils/codecs.rb', line 366

def encode_smallvec_u8_pubkeys(pubkeys)
  [pubkeys.length] + pubkeys.flat_map { |pubkey| encode_pubkey(pubkey) }
end

#encode_u8(byte) ⇒ Array<Integer>

Encodes a u8 as a single byte.

Parameters:

  • byte (Integer) —

    Value in range 0..255.

Returns:

  • (Array<Integer>) —

    A single-element byte array.

Since:

  • 0.0.1



114
115
116
 
# File 'lib/solace/utils/codecs.rb', line 114

def encode_u8(byte)
  [byte]
end

#encode_vec_pubkeys(pubkeys) ⇒ Array<Integer>

Encodes a Borsh Vec<publicKey>: u32 LE count prefix followed by each 32-byte pubkey.

Parameters:

  • pubkeys (Array<#to_s>) —

    The public keys in any representation.

Returns:

  • (Array<Integer>)

Since:

  • 0.0.1



345
346
347
348
 
# File 'lib/solace/utils/codecs.rb', line 345

def encode_vec_pubkeys(pubkeys)
  encode_le_u32(pubkeys.length).bytes +
    pubkeys.flat_map { |pubkey| encode_pubkey(pubkey) }
end

#valid_base58?(string) ⇒ Boolean

Checks if a string is a valid Base58 string

Parameters:

  • string (String) —

    The string to check

Returns:

  • (Boolean) —

    True if the string is a valid Base58 string, false otherwise

Since:

  • 0.0.1



99
100
101
102
103
104
105
106
 
# File 'lib/solace/utils/codecs.rb', line 99

def valid_base58?(string)
  return false if string.nil? || string.empty?

  Base58.decode(string)
  true
rescue StandardError => _e
  false
end