Module: RuboCop::AST::PrismLazyTokens Private

Defined in:
lib/rubocop/ast/processed_source.rb

Overview

This module is part of a private API. You should avoid using this module if possible, as it may be removed or be changed in the future.

Extends the prism translation parsers so that the conversion of tokens into the parser gem's format is deferred until the tokens are first accessed. Building the tokens is a significant part of the translation cost, and not every caller needs them.

Instance Method Summary collapse

Instance Method Details

#tokenize_deferred(source_buffer) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Same contract as Parser::Base#tokenize, except the tokens are returned as a callable that performs the conversion when invoked.



36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/rubocop/ast/processed_source.rb', line 36

def tokenize_deferred(source_buffer)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result = unwrap(@parser.parse_lex(source, **prism_options), offset_cache)

  program, tokens = result.value
  ast = build_ast(program, offset_cache) if result.success?
  comments = build_comments(result.comments, offset_cache)

  [ast, comments, deferred_tokens(source_buffer, tokens, offset_cache)]
ensure
  @source_buffer = nil
end