Module: Pikuri::Lsp::PositionEncoding

Defined in:
lib/pikuri/lsp/position_encoding.rb

Overview

LSP's positionEncoding values and the offset arithmetic each one implies. A column is a character index inside pikuri and a code-unit offset on the wire; column 12 of the same line is three different numbers depending on who is asking:

line = 'x = "héllo🎉"'                        # 'é' is 2 bytes, '🎉' is 4
PositionEncoding.offset_for(line, 12, UTF32)  # => 11  (just 0-based)
PositionEncoding.offset_for(line, 12, UTF16)  # => 12  (the emoji counts twice)
PositionEncoding.offset_for(line, 12, UTF8)   # => 15
PositionEncoding.column_for(line, 12, UTF16)  # => 12  (round-trips)

Converting in the wrong direction, or not at all, yields a plausible column — so the server answers confidently about the wrong token instead of failing. DEFAULT is what a server that negotiates nothing means.

Constant Summary collapse

UTF8 =

Offsets count UTF-8 bytes. ruby-lsp accepts pikuri's offer of this.

'utf-8'
UTF16 =

Offsets count UTF-16 code units, so an astral character (emoji, rare CJK) counts twice. jdtls ignores the offer and stays here.

'utf-16'
UTF32 =

Offsets count codepoints, i.e. exactly pikuri's own columns.

'utf-32'
DEFAULT =

What a server that answers no positionEncoding means: UTF-16 is the 3.17 default, and the conversion is therefore mandatory code rather than a fallback for a hypothetical server.

UTF16
OFFERED =

Encodings pikuri offers in general.positionEncodings, best first. UTF-32 is out: no measured server offers it, and it would be one more branch nothing exercises.

[UTF8, UTF16].freeze
SUPPORTED =

Every value a caller may pass.

[UTF8, UTF16, UTF32].freeze

Class Method Summary collapse

Class Method Details

.column_for(line_text, offset, encoding) ⇒ Integer

1-based character column for a wire offset into line_text.

An offset past the end of the line clamps to one-past-the-last character rather than raising: a range's end legitimately sits there, and a server that overshoots is relayed, not fought.

Parameters:

  • line_text (String, nil) —

    the line the offset points into, or nil when the text is not to hand — the offset is then read as a character count, exact for an all-ASCII line and off by one per preceding multi-unit character otherwise. Locations a server walked to arrive this way, and only their line number gets rendered.

  • offset (Integer) —

    0-based offset in +encoding+'s code units.

  • encoding (String) —

    one of SUPPORTED.

Returns:

  • (Integer) —

    1-based character column.

Raises:

  • (ArgumentError) —

    if offset is negative or encoding is unknown.



91
92
93
94
95
96
97
98
99
100
101
102
103
 
# File 'lib/pikuri/lsp/position_encoding.rb', line 91

def column_for(line_text, offset, encoding)
  raise ArgumentError, "offset must be >= 0, got #{offset}" if offset.negative?
  raise ArgumentError, "unknown positionEncoding #{encoding.inspect}" unless SUPPORTED.include?(encoding)
  return offset + 1 if line_text.nil? || encoding == UTF32

  chars =
    if encoding == UTF8
      (line_text.byteslice(0, offset) || line_text).scrub.length
    else
      chars_before_utf16(line_text, offset)
    end
  chars + 1
end

.offset_for(line_text, column, encoding) ⇒ Integer

Wire offset for a 1-based character column on line_text.

offset_for('def résolve', 8, UTF8)   # => 8  ("def rés" is 8 bytes)

Parameters:

  • line_text (String) —

    the line the column points into; a trailing newline is harmless.

  • column (Integer) —

    1-based character column, as Pikuri::Lsp::Position holds it.

  • encoding (String) —

    one of SUPPORTED.

Returns:

  • (Integer) —

    0-based offset in that encoding's code units.

Raises:

  • (ArgumentError) —

    if column is below 1, points past the end of line_text (pikuri derived a column its own line cannot hold — a bug in the caller, not a server quirk), or encoding is unknown.



57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/pikuri/lsp/position_encoding.rb', line 57

def offset_for(line_text, column, encoding)
  raise ArgumentError, "column must be >= 1, got #{column}" if column < 1

  chars = column - 1
  if chars > line_text.length
    raise ArgumentError,
          "column #{column} is past the end of a #{line_text.length}-character " \
          "line: #{line_text.inspect}"
  end

  prefix = line_text[0, chars]
  case encoding
  when UTF32 then chars
  when UTF8  then prefix.bytesize
  when UTF16 then utf16_units(prefix)
  else raise ArgumentError, "unknown positionEncoding #{encoding.inspect}"
  end
end

.utf16_units(str) ⇒ Integer

UTF-16 code units str occupies.

Parameters:

  • str (String)

Returns:

  • (Integer) 


109
110
111
 
# File 'lib/pikuri/lsp/position_encoding.rb', line 109

def utf16_units(str)
  str.each_char.sum { |ch| ch.valid_encoding? && ch.ord > 0xFFFF ? 2 : 1 }
end