Class: LexerKit::Format::LKB1

Inherits:
Object
  • Object
show all
Defined in:
lib/lexer_kit/format/lkb1.rb,
lib/lexer_kit/format/lkb1/decoder.rb

Overview

LKB1 (LexerKit Binary format version 1) is a binary container format for compiled lexer programs. It provides efficient loading with optional metadata and integrity checking via SHA256.

Defined Under Namespace

Classes: Decoder

Constant Summary collapse

MAGIC = 
"LKB1"
HEADER_VERSION = 
4
FIXED_HEADER_LEN = 
52
FLAG_PAYLOAD_COMPRESSED = 
1 << 0
FLAG_PAYLOAD_ENCRYPTED = 
1 << 1
FLAG_PAYLOAD_SHA256 = 
1 << 2
FLAG_META = 
1 << 3
TLV_BUILD_ID = 
0x0001
TLV_SOURCE_VERSION = 
0x0002
TLV_TOKEN_COUNT = 
0x0003
TLV_MODE_COUNT = 
0x0004
TLV_INSTRUCTION_COUNT = 
0x0005
TLV_CREATED_AT = 
0x0006
TLV_GENERATOR_VERSION = 
0x0007
TLV_MAX_STRING_LENGTH =

TLV length is u16, so string data cannot exceed this

(2**16) - 1

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(program, meta: {}) ⇒ LKB1

Create a new LKB1 instance from a compiled program

Parameters:

  • program (IR::CompiledProgram) —

    compiled lexer program

  • meta (Hash) (defaults to: {}) —

    optional metadata



37
38
39
40
 
# File 'lib/lexer_kit/format/lkb1.rb', line 37

def initialize(program, meta: {})
  @program = program
  @meta = meta
end

Instance Attribute Details

#meta ⇒ Object (readonly)

Returns the value of attribute meta.



32
33
34
 
# File 'lib/lexer_kit/format/lkb1.rb', line 32

def meta
  @meta
end

#program ⇒ Object (readonly)

Returns the value of attribute program.



32
33
34
 
# File 'lib/lexer_kit/format/lkb1.rb', line 32

def program
  @program
end

Class Method Details

.decode(bytes) ⇒ LKB1

Decode a compiled lexer program from lkb1 binary data

Parameters:

  • bytes (String) —

    binary data

Returns:

  • (LKB1) —

    LKB1 instance



61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/lexer_kit/format/lkb1.rb', line 61

def self.decode(bytes)
  decoded = Decoder.new(bytes).decode_container
  header = decoded[:header]

  unless header[:format_version] == IR::Serializer::FORMAT_VERSION
    raise LexerKit::IntegrityError, "Unsupported format version: #{header[:format_version]}"
  end

  program = IR::CompiledProgram.from_binary(decoded[:payload])
  program.load_native! if LexerKit.native?
  new(program, meta: decoded[:meta])
rescue ArgumentError => e
  raise LexerKit::IntegrityError, e.message
end

.load(path) ⇒ LKB1

Load a compiled lexer program from a .lkb1 file

Parameters:

  • path (String) —

    path to .lkb1 file

Returns:

  • (LKB1) —

    LKB1 instance



45
46
47
48
 
# File 'lib/lexer_kit/format/lkb1.rb', line 45

def self.load(path)
  bytes = File.binread(path)
  decode(bytes)
end

.save(program, path:, meta: {}) ⇒ Object

Save a compiled lexer program to a .lkb1 file (shortcut)

Parameters:

  • program (IR::CompiledProgram) —

    compiled lexer program

  • path (String) —

    output file path

  • meta (Hash) (defaults to: {}) —

    optional metadata for header



54
55
56
 
# File 'lib/lexer_kit/format/lkb1.rb', line 54

def self.save(program, path:, meta: {})
  new(program, meta: meta).save(path)
end

Instance Method Details

#encode ⇒ String

Encode the program to lkb1 binary data

Returns:

  • (String) —

    binary data



78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/lexer_kit/format/lkb1.rb', line 78

def encode
  payload = @program.to_binary

  # Auto-generate metadata from program
  default_meta = {
    token_count: @program.token_names.size,
    mode_count: @program.mode_names.size,
    instruction_count: @program.instructions.size,
    created_at: Time.now.to_i,
    generator_version: LexerKit::VERSION
  }

  encode_container(
    payload,
    meta: default_meta.merge(@meta),
    format_version: IR::Serializer::FORMAT_VERSION
  )
end

#save(path) ⇒ Object

Save the program to a .lkb1 file

Parameters:

  • path (String) —

    output file path



99
100
101
 
# File 'lib/lexer_kit/format/lkb1.rb', line 99

def save(path)
  File.binwrite(path, encode)
end