Class: TreeHaver::GrammarFinder

Inherits:
Object
  • Object
show all
Defined in:
lib/tree_haver/grammar_finder.rb

Overview

Registration-first utility for finding tree-sitter grammar shared libraries.

GrammarFinder resolves tree-sitter grammars in a constrained order:

  1. explicit environment override
  2. existing TreeHaver registration
  3. explicit extra paths
  4. tree_sitter_language_pack parser backend registration

This class is designed to be used by language-specific merge gems without requiring TreeHaver to own parser- or grammar-specific policy.

Security Considerations

Loading shared libraries is inherently dangerous as it executes arbitrary native code. GrammarFinder performs the following security validations:

  • Language names are validated to contain only safe characters
  • Paths from environment variables are validated before use
  • Path traversal attempts (../) are rejected
  • Only files with expected extensions (.so, .dylib, .dll) are accepted

For additional security, use #find_library_path_safe which only returns paths from trusted system directories.

Examples:

Basic usage

finder = TreeHaver::GrammarFinder.new(:toml)
path = finder.find_library_path
# => "/usr/local/lib/libtree_sitter_toml.so"

Check availability

finder = TreeHaver::GrammarFinder.new(:json)
if finder.available?
  language = TreeHaver::Language.load(finder.language_name, finder.find_library_path)
end

Register with TreeHaver

finder = TreeHaver::GrammarFinder.new(:bash)
finder.register! if finder.available?
# Now you can use: TreeHaver::Language.bash

With custom search paths

finder = TreeHaver::GrammarFinder.new(:toml, extra_paths: ["/opt/custom/lib"])

Secure mode (trusted directories only)

finder = TreeHaver::GrammarFinder.new(:toml)
path = finder.find_library_path_safe  # Only returns paths in trusted dirs

See Also:

Constant Summary collapse

TREE_SITTER_BACKENDS =

Backends that use tree-sitter (require native runtime libraries) Other backends (Citrus, Prism, Psych, etc.) don't use tree-sitter

[
  TreeHaver::Backends::MRI,
  TreeHaver::Backends::FFI,
  TreeHaver::Backends::Rust,
  TreeHaver::Backends::Java
].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(language_name, extra_paths: [], validate: true) ⇒ GrammarFinder

Initialize a grammar finder for a specific language

Parameters:

  • language_name (Symbol, String) —

    the tree-sitter language name (e.g., :toml, :json, :bash)

  • extra_paths (Array<String>) (defaults to: []) —

    additional paths to search (searched first after ENV)

  • validate (Boolean) (defaults to: true) —

    if true, validates the language name (default: true)

Raises:

  • (ArgumentError) —

    if language_name is invalid and validate is true



68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/tree_haver/grammar_finder.rb', line 68

def initialize(language_name, extra_paths: [], validate: true)
  name_str = language_name.to_s.downcase

  if validate && !PathValidator.safe_language_name?(name_str)
    raise ArgumentError, "Invalid language name: #{language_name.inspect}. " \
      'Language names must start with a letter and contain only lowercase letters, numbers, and underscores.'
  end

  @language_name = name_str.to_sym
  @extra_paths = Array(extra_paths)
end

Instance Attribute Details

#extra_paths ⇒ Array<String> (readonly)

Returns additional search paths provided at initialization.

Returns:

  • (Array<String>) —

    additional search paths provided at initialization



60
61
62
 
# File 'lib/tree_haver/grammar_finder.rb', line 60

def extra_paths
  @extra_paths
end

#language_name ⇒ Symbol (readonly)

Returns the language identifier.

Returns:

  • (Symbol) —

    the language identifier



57
58
59
 
# File 'lib/tree_haver/grammar_finder.rb', line 57

def language_name
  @language_name
end

Class Method Details

.reset_runtime_check! ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Reset the cached tree-sitter runtime check (for testing)



294
295
296
 
# File 'lib/tree_haver/grammar_finder.rb', line 294

def reset_runtime_check!
  remove_instance_variable(:@tree_sitter_runtime_usable) if defined?(@tree_sitter_runtime_usable)
end

.tree_sitter_runtime_usable? ⇒ Boolean

Check if the tree-sitter runtime is usable

Tests whether we can actually create a tree-sitter parser. Result is cached since this is expensive and won't change during runtime.

Returns:

  • (Boolean) —

    true if tree-sitter runtime is functional



269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
 
# File 'lib/tree_haver/grammar_finder.rb', line 269

def tree_sitter_runtime_usable?
  return @tree_sitter_runtime_usable if defined?(@tree_sitter_runtime_usable)

  @tree_sitter_runtime_usable = begin
    # Try to create a parser using the current backend
    mod = TreeHaver.resolve_backend_module(nil)

    # Only tree-sitter backends are relevant here
    # Non-tree-sitter backends (Citrus, Prism, Psych, etc.) don't use grammar files
    if mod.nil? || !TREE_SITTER_BACKENDS.include?(mod)
      false
    else
      # Try to instantiate a parser - this will fail if runtime isn't available
      mod::Parser.new
      true
    end
  rescue NoMethodError, LoadError, NotAvailable => _e
    # NOTE: FFI::NotFoundError inherits from LoadError, so it's caught here too
    false
  end
end

Instance Method Details

#available? ⇒ Boolean

Check if the grammar library is available AND usable

This checks:

  1. The grammar library file exists
  2. The tree-sitter runtime is functional (can create a parser)

This prevents registering grammars when tree-sitter isn't actually usable, allowing clean fallback to alternative backends like Citrus.

Returns:

  • (Boolean) —

    true if the library can be found AND tree-sitter runtime works



242
243
244
245
246
247
248
249
250
251
 
# File 'lib/tree_haver/grammar_finder.rb', line 242

def available?
  return true if tree_sitter_language_pack_parser_available?

  path = find_library_path
  return false if path.nil?

  # Check if tree-sitter runtime is actually functional
  # This is cached at the class level since it's the same for all grammars
  self.class.tree_sitter_runtime_usable?
end

#available_safe? ⇒ Boolean

Check if the grammar library is available in a trusted directory

Returns:

  • (Boolean) —

    true if the library can be found in a trusted directory

See Also:



303
304
305
 
# File 'lib/tree_haver/grammar_finder.rb', line 303

def available_safe?
  !find_library_path_safe.nil?
end

#env_var_name ⇒ String

Get the environment variable name for this language

Returns:

  • (String) —

    the ENV var name (e.g., "TREE_SITTER_TOML_PATH")



83
84
85
 
# File 'lib/tree_haver/grammar_finder.rb', line 83

def env_var_name
  "TREE_SITTER_#{@language_name.to_s.upcase}_PATH"
end

#find_library_path ⇒ String?

Note:

Paths from ENV are validated using PathValidator.safe_library_path? to prevent path traversal and other attacks. Invalid ENV paths cause an error to be raised (Principle of Least Surprise - explicit paths must work).

Note:

Setting the ENV variable to an empty string explicitly disables this grammar. This allows fallback to alternative backends (e.g., Citrus).

Find the grammar library path

Searches in order:

  1. Environment variable override (validated for safety)
  2. Existing TreeHaver tree-sitter registration
  3. Extra paths provided at initialization tree_sitter_language_pack is intentionally not exposed as a shared-library path fallback here. It is registered as a TreeHaver backend module when its parser API is available.

Returns:

  • (String, nil) —

    the path to the library, or nil if not found

Raises:

See Also:



155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
 
# File 'lib/tree_haver/grammar_finder.rb', line 155

def find_library_path
  # Check environment variable first (highest priority)
  # Use key? to distinguish between "not set" and "set to empty"
  env_var = env_var_name
  if ENV[env_var] || ENV.key?(env_var)
    env_path = ENV[env_var]

    # simplecov:disable defensive - ENV.key? true with nil value is rare edge case
    if env_path.nil?
      @env_rejection_reason = 'explicitly disabled (set to nil)'
      return
    end
    # simplecov:enable

    # Empty string means "explicitly skip this grammar"
    # This allows users to disable tree-sitter for specific languages
    # and fall back to alternative backends like Citrus
    if env_path.empty?
      @env_rejection_reason = 'explicitly disabled (set to empty string)'
      return
    end

    # Store why env path was rejected for better error messages
    @env_rejection_reason = validate_env_path(env_path)

    # Principle of Least Surprise: If user explicitly sets an ENV variable
    # to a path, that path MUST work. Don't silently fall back to auto-discovery.
    if @env_rejection_reason
      raise TreeHaver::NotAvailable,
            "#{env_var_name} is set to #{env_path.inspect} but #{@env_rejection_reason}. " \
              'Either fix the path, unset the variable to use auto-discovery, ' \
              'or set it to empty string to explicitly disable this grammar.'
    end

    return env_path
  end

  registered_path = registered_tree_sitter_path
  return registered_path if registered_path

  explicit_path = explicit_search_path
  return explicit_path if explicit_path

  nil
end

#find_library_path_safe ⇒ String?

Find the grammar library path with strict security validation

This method only returns paths that are in trusted system directories. Use this when you want maximum security and don't need to support custom installation locations.

Returns:

  • (String, nil) —

    the path to the library, or nil if not found

See Also:

  • For the list of trusted directories


226
227
228
229
230
 
# File 'lib/tree_haver/grammar_finder.rb', line 226

def find_library_path_safe
  search_paths.find do |path|
    File.exist?(path) && PathValidator.in_trusted_directory?(path)
  end
end

#library_filename ⇒ String

Get the canonical tree-sitter-language-pack filename for the current platform

Returns:

  • (String) —

    the library filename (e.g., "libtree_sitter_toml.so")



97
98
99
 
# File 'lib/tree_haver/grammar_finder.rb', line 97

def library_filename
  library_filenames.first
end

#library_filenames ⇒ Array<String>

Get all accepted library filenames for this language

Accept both the tree-sitter-language-pack naming convention and the historical hyphenated form used by some standalone grammar builds.

Returns:

  • (Array<String>) 


107
108
109
110
111
112
113
 
# File 'lib/tree_haver/grammar_finder.rb', line 107

def library_filenames
  ext = platform_extension
  [
    "libtree_sitter_#{@language_name}#{ext}",
    "libtree-sitter-#{@language_name}#{ext}"
  ]
end

#not_found_message ⇒ String

Get a human-readable error message when library is not found

Returns:

  • (String) —

    error message with installation hints



360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
 
# File 'lib/tree_haver/grammar_finder.rb', line 360

def not_found_message
  msg = "tree-sitter #{@language_name} grammar not found."

  # Check if env var is set but rejected
  env_value = ENV[env_var_name]
  msg += if env_value && @env_rejection_reason
           " #{env_var_name} is set to #{env_value.inspect} but #{@env_rejection_reason}."
         elsif env_value && File.exist?(env_value) && !self.class.tree_sitter_runtime_usable?
           " #{env_var_name} is set and file exists, but no tree-sitter runtime is available. " \
             'Add ruby_tree_sitter, ffi, or tree_stump gem to your Gemfile.'
         else
           " Searched: #{search_paths.join(', ')}."
         end

  msg + ' Register the grammar, install tree_sitter_language_pack with parser API support, ' \
        "or set #{env_var_name} to a valid path."
end

#register!(raise_on_missing: false) ⇒ Boolean

Register this language with TreeHaver

After registration, the language can be loaded via dynamic method (e.g., TreeHaver::Language.toml).

Parameters:

  • raise_on_missing (Boolean) (defaults to: false) —

    if true, raises when library not found

Returns:

  • (Boolean) —

    true if registration succeeded

Raises:

  • (NotAvailable) —

    if library not found and raise_on_missing is true



315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
 
# File 'lib/tree_haver/grammar_finder.rb', line 315

def register!(raise_on_missing: false)
  if tree_sitter_language_pack_parser_available?
    TreeHaver.register_language(
      @language_name,
      backend_module: TreeHaver::Backends::Tslp,
      backend_type: :tslp,
      gem_name: 'tree_sitter_language_pack'
    )
    return true
  end

  path = find_library_path
  unless path
    raise NotAvailable, not_found_message if raise_on_missing

    return false
  end

  TreeHaver.register_language(@language_name, path: path, symbol: symbol_name)
  true
end

#search_info ⇒ Hash

Get debug information about the search

Returns:

  • (Hash) —

    diagnostic information



340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
 
# File 'lib/tree_haver/grammar_finder.rb', line 340

def search_info
  found = find_library_path # This populates @env_rejection_reason
  {
    language: @language_name,
    env_var: env_var_name,
    env_value: ENV[env_var_name],
    env_rejection_reason: @env_rejection_reason,
    tree_sitter_language_pack_parser_available: tree_sitter_language_pack_parser_available?,
    symbol: symbol_name,
    library_filename: library_filename,
    library_filenames: library_filenames,
    search_paths: search_paths,
    found_path: found,
    available: tree_sitter_language_pack_parser_available? || !found.nil?
  }
end

#search_paths ⇒ Array<String>

Generate the full list of search paths for this language

Order: registered path, then explicit extra paths.

Returns:

  • (Array<String>) —

    all paths to search



120
121
122
123
124
125
126
127
128
129
130
131
132
133
 
# File 'lib/tree_haver/grammar_finder.rb', line 120

def search_paths
  paths = []

  registration = registered_tree_sitter_registration
  paths << registration[:path] if registration&.dig(:path)

  @extra_paths.each do |dir|
    library_filenames.each do |filename|
      paths << File.join(dir, filename)
    end
  end

  paths.uniq
end

#symbol_name ⇒ String

Get the expected symbol name exported by the grammar library

Returns:

  • (String) —

    the symbol name (e.g., "tree_sitter_toml")



90
91
92
 
# File 'lib/tree_haver/grammar_finder.rb', line 90

def symbol_name
  "tree_sitter_#{@language_name}"
end

#validate_env_path(path) ⇒ String?

Validate an environment variable path and return reason if invalid

Returns:

  • (String, nil) —

    rejection reason or nil if valid



203
204
205
206
207
208
209
210
211
212
213
214
215
216
 
# File 'lib/tree_haver/grammar_finder.rb', line 203

def validate_env_path(path)
  # Check for leading/trailing whitespace
  return "contains leading or trailing whitespace (use #{path.strip.inspect})" if path != path.strip

  # Check if path is safe
  unless PathValidator.safe_library_path?(path)
    return 'failed security validation (may contain path traversal or suspicious characters)'
  end

  # Check if file exists
  return 'file does not exist' unless File.exist?(path)

  nil # Valid!
end