Module: Oselvar::Var::Core::Offsets

Defined in:
lib/oselvar/var/core/span.rb

Overview

UTF-16 offset conversion. Ruby strings are code-point indexed, so the whole core converts to/from UTF-16 code units here (the single riskiest part of the port — mirrors Python's span.py). Reused by the matcher and by hash.rb.

Class Method Summary collapse

Class Method Details

.cp_index_for_utf16(source, u16) ⇒ Object

Inverse of to_utf16_offset: the code-point index at a UTF-16 offset.



35
36
37
38
39
40
41
42
43
 
# File 'lib/oselvar/var/core/span.rb', line 35

def cp_index_for_utf16(source, u16)
  count = 0
  source.each_char.with_index do |ch, i|
    return i if count >= u16

    count += ch.ord > 0xFFFF ? 2 : 1
  end
  source.length
end

.line_col(source, offset_u16) ⇒ Object

1-based [line, col] at a UTF-16 offset; col counts UTF-16 units and resets to 1 after each newline (mirrors span.ts's lineCol).



54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
 
# File 'lib/oselvar/var/core/span.rb', line 54

def line_col(source, offset_u16)
  line = 1
  col = 1
  count = 0
  source.each_char do |ch|
    break if count >= offset_u16

    width = ch.ord > 0xFFFF ? 2 : 1
    if ch == "\n"
      line += 1
      col = 1
    else
      col += width
    end
    count += width
  end
  [line, col]
end

.span_from_offsets(source, start_u16, end_u16) ⇒ Object

Build a Span from UTF-16 start/end offsets.



74
75
76
77
78
79
80
81
82
 
# File 'lib/oselvar/var/core/span.rb', line 74

def span_from_offsets(source, start_u16, end_u16)
  start_line, start_col = line_col(source, start_u16)
  end_line, end_col = line_col(source, end_u16)
  Span.new(
    start_offset: start_u16, end_offset: end_u16,
    start_line: start_line, start_col: start_col,
    end_line: end_line, end_col: end_col
  )
end

.to_utf16_offset(source, cp_index) ⇒ Object

UTF-16 offset of the code-point index cp_index in source.



30
31
32
 
# File 'lib/oselvar/var/core/span.rb', line 30

def to_utf16_offset(source, cp_index)
  utf16_len(source[0...cp_index])
end

.utf16_len(str) ⇒ Object

UTF-16 code-unit length of a string (astral chars count as 2).



23
24
25
26
27
 
# File 'lib/oselvar/var/core/span.rb', line 23

def utf16_len(str)
  n = 0
  str.each_char { |ch| n += ch.ord > 0xFFFF ? 2 : 1 }
  n
end

.utf16_slice(source, start_u16, end_u16) ⇒ Object

Slice source by UTF-16 offsets, returning the covered substring.



46
47
48
49
50
 
# File 'lib/oselvar/var/core/span.rb', line 46

def utf16_slice(source, start_u16, end_u16)
  a = cp_index_for_utf16(source, start_u16)
  b = cp_index_for_utf16(source, end_u16)
  source[a...b]
end