Module: Scryer::Ast

Defined in:
lib/scryer/ast.rb

Overview

Small set of helpers for walking the S-expression tree that Ripper.sexp produces. We deliberately don't depend on the parser/RuboCop::AST gems so this gem has zero runtime dependencies beyond Ruby's own stdlib — Ripper has shipped with Ruby since 1.9.

A Ripper sexp node is either a plain Ruby object (String/Integer/nil/false) or an Array whose first element is a Symbol tag (:def, :call, :string_literal, etc.) followed by child nodes. Terminal "token" nodes look like [:@ident, "foo", [line, col]] — the trailing [line, col] pair is what lets us report accurate line numbers for findings.

Class Method Summary collapse

Class Method Details

.call_arguments(node) ⇒ Object

Given a [:method_add_arg, call_node, args_node] or [:command, ident, args_node] node, return the flattened list of top-level argument sexp nodes (best effort — walks through the [:arg_paren, [:args_add_block, [args...], block]] wrapping).



97
98
99
100
101
102
103
104
105
106
107
108
 
# File 'lib/scryer/ast.rb', line 97

def call_arguments(node)
  return [] unless node.is_a?(Array)

  args_node =
    case node[0]
    when :method_add_arg then node[2]
    when :command then node[2]
    when :command_call then node[4]
    end

  unwrap_args(args_node)
end

.call_name(node) ⇒ Object

Matches a .method_name(...) or bare method_name(...) call node. Returns the receiver node (nil for a bare/vcall) and the method name string if node is a call to one of method_names, else nil.

Handles the shapes Ripper produces for a called method:

[:call, receiver, [:@period,...]|:"::", [:@ident, "name", pos]]        (has a receiver, parens or no args)
[:vcall, [:@ident, "name", pos]]                                       (bare, no args)
[:fcall, [:@ident, "name", pos]]                                       (bare + parens, no receiver)
[:command, [:@ident, "name", pos], args]                               (bare, with args, no parens)
[:command_call, receiver, [:@period,...], [:@ident, "name", pos], args] (receiver + args, no parens —
                                                                       e.g. `config.session_store :x, y: z`)
[:method_add_arg, call_or_fcall_node, args_node]                       (receiver/bare + parens, wraps one of the above)


75
76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/scryer/ast.rb', line 75

def call_name(node)
  case node
  when ->(n) { tagged?(n, :call, :command_call) }
    [node[1], ident_text(node[3])]
  when ->(n) { tagged?(n, :vcall) }
    [nil, ident_text(node[1])]
  when ->(n) { tagged?(n, :fcall) }
    [nil, ident_text(node[1])]
  when ->(n) { tagged?(n, :command) }
    [nil, ident_text(node[1])]
  end
end

.closing_interpolation_braces(line, from_col, node) ⇒ Object 



235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
 
# File 'lib/scryer/ast.rb', line 235

def closing_interpolation_braces(line, from_col, node)
  return "" unless line

  path = path_to_last_token(node)
  count = path ? path.count { |n| tagged?(n, :string_embexpr, :string_dvar) } : 0
  return "" if count.zero?

  cursor = from_col
  result = +""
  count.times do
    break unless line[cursor] == "}"

    result << "}"
    cursor += 1
  end
  result
end

.each_node(node, &block) ⇒ Object

Depth-first walk of every node in the tree. Yields each node (both tagged Array nodes and plain values) to the block. This is intentionally simple/generic rather than type-specific, so new rules can filter for whatever node shape they care about.



21
22
23
24
25
26
27
28
29
30
 
# File 'lib/scryer/ast.rb', line 21

def each_node(node, &block)
  return enum_for(:each_node, node) unless block

  block.call(node)
  return unless node.is_a?(Array)

  node.each do |child|
    each_node(child, &block) if child.is_a?(Array)
  end
end

.exact_source_text(source, node) ⇒ Object

Column-precise source text for exactly what node spans — unlike source_text, doesn't pull in the rest of the line. Needed for e.g. a cache key expression that shares a line with the surrounding assignment/call (x = Rails.cache.fetch("key_#{id}") { ... } unless x) — source_text would return that whole statement, not just the key. Built from the node's first/last terminal tokens (assumed to appear in source order, which holds for the expressions this is used on); nil if the node has no terminal tokens or its positions don't fit the source.

Ripper.sexp doesn't emit a terminal token for a string interpolation's closing } — if the interpolation is the last thing in the string ("foo_#{id}"), the raw span above ends right after id, one } short of the true end. closing_interpolation_braces counts how many string_embexpr/string_dvar wrappers actually contain that last token (usually 0 or 1, more if nested) and appends exactly that many } — only when the source really has one there, never guessed blindly.



187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
 
# File 'lib/scryer/ast.rb', line 187

def exact_source_text(source, node)
  tokens = each_node(node).select do |n|
    n.is_a?(Array) && n[0].is_a?(Symbol) && n[0].to_s.start_with?("@") &&
      n[2].is_a?(Array) && n[2].size == 2
  end
  return nil if tokens.empty?

  lines = source.lines
  start_line, start_col = tokens.first[2]
  end_line, end_col = tokens.last[2]
  end_col += tokens.last[1].to_s.length
  return nil if start_line.nil? || end_line.nil? || end_line > lines.size

  text =
    if start_line == end_line
      lines[start_line - 1][start_col...end_col]
    else
      ([lines[start_line - 1][start_col..]] +
        lines[start_line...(end_line - 1)] +
        [lines[end_line - 1][0...end_col]]).join
    end
  return nil unless text

  text + closing_interpolation_braces(lines[end_line - 1], end_col, node)
end

.false_literal?(node) ⇒ Boolean

Returns:

  • (Boolean) 


289
290
291
 
# File 'lib/scryer/ast.rb', line 289

def false_literal?(node)
  kw_literal?(node) == "false"
end

.ident_text(node) ⇒ Object 



88
89
90
91
92
 
# File 'lib/scryer/ast.rb', line 88

def ident_text(node)
  return nil unless node.is_a?(Array)

  node[1] if node[0].is_a?(Symbol) && %i[@ident @const @kw @op].include?(node[0])
end

.keyword_arg(args, label) ⇒ Object

Given a list of top-level argument nodes (from call_arguments), finds the bare_assoc_hash entry whose label matches label: and returns its value node, or nil if that keyword argument isn't present. Covers Rails DSL calls like session_store :cookie_store, secure: true or http_basic_authenticate_with password: "x", where keyword args arrive as a bare_assoc_hash rather than a real Hash literal.



259
260
261
262
263
264
265
266
267
268
269
270
 
# File 'lib/scryer/ast.rb', line 259

def keyword_arg(args, label)
  target = "#{label}:"

  args.each do |arg|
    next unless tagged?(arg, :bare_assoc_hash) && arg[1].is_a?(Array)

    pair = arg[1].find { |p| tagged?(p, :assoc_new) && p[1].is_a?(Array) && p[1][0] == :@label && p[1][1] == target }
    return pair[2] if pair
  end

  nil
end

.kw_literal?(node) ⇒ Boolean

Returns:

  • (Boolean) 


293
294
295
296
297
 
# File 'lib/scryer/ast.rb', line 293

def kw_literal?(node)
  return nil unless tagged?(node, :var_ref) && node[1].is_a?(Array) && node[1][0] == :@kw

  node[1][1]
end

.line_of(node) ⇒ Object 



59
60
61
 
# File 'lib/scryer/ast.rb', line 59

def line_of(node)
  position_of(node)&.first
end

.line_range_of(node) ⇒ Object

The [start_line, end_line] a node spans, found by scanning its descendants for position-bearing terminal tokens (nodes themselves don't carry an explicit end line — see MethodExtractor's comment). Returns nil if the node has no position-bearing descendants at all.



151
152
153
154
155
156
157
 
# File 'lib/scryer/ast.rb', line 151

def line_range_of(node)
  positions = each_node(node).filter_map { |n| position_of(n) }
  return nil if positions.empty?

  lines = positions.map(&:first)
  [lines.min, lines.max]
end

.literal_text(node) ⇒ Object

The literal text of a symbol_literal (:inline) or plain string_literal ("inline") node — the two shapes a Rails DSL keyword argument's value commonly takes. nil for anything else (interpolated string, array, boolean, ...).



276
277
278
279
280
281
 
# File 'lib/scryer/ast.rb', line 276

def literal_text(node)
  return plain_string_value(node) if tagged?(node, :string_literal)
  return nil unless tagged?(node, :symbol_literal) && node[1].is_a?(Array)

  ident_text(node[1][1]) if tagged?(node[1], :symbol)
end

.normalized_tokens(node) ⇒ Object

Walks every terminal token-bearing node inside node and maps it to a normalized symbol: identifiers/literals become placeholders (so renamed variables/changed literal values still count as "the same" shape), keywords/operators/punctuation stay literal (so the actual control-flow shape of the code still has to match for two nodes to look similar). Shared by MethodExtractor, QueryExtractor, and CacheExtractor — anything that needs to compare two code fragments for near-duplication.



320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
 
# File 'lib/scryer/ast.rb', line 320

def normalized_tokens(node)
  each_node(node).filter_map do |n|
    next unless n.is_a?(Array) && n[0].is_a?(Symbol) && n[0].to_s.start_with?("@")

    case n[0]
    when :@ident, :@const, :@ivar, :@gvar, :@cvar, :@label
      :ID
    when :@int, :@float, :@CHAR
      :LIT_NUM
    when :@tstring_content
      :LIT_STR
    when :@kw
      n[1].to_sym # if/else/end/def/do/while/... — structurally meaningful
    when :@op, :@period, :@comma, :@lbracket, :@rbracket, :@lparen, :@rparen,
         :@lbrace, :@rbrace, :@semicolon
      n[1].to_sym
    end
  end
end

.path_to_last_token(node) ⇒ Object

The ancestor chain (node itself first) from node down to whichever descendant holds the last terminal token found by depth-first order — i.e. mirrors each_node's traversal, but keeps the path instead of just the leaf, so callers can inspect what wraps that last token.



217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
 
# File 'lib/scryer/ast.rb', line 217

def path_to_last_token(node)
  return nil unless node.is_a?(Array)

  if node[0].is_a?(Symbol) && node[0].to_s.start_with?("@") && node[2].is_a?(Array) && node[2].size == 2
    return [node]
  end

  last_path = nil
  node.each do |child|
    next unless child.is_a?(Array)

    sub = path_to_last_token(child)
    last_path = sub if sub
  end

  last_path && [node] + last_path
end

.plain_string_value(node) ⇒ Object

Extracts the literal text of a plain (non-interpolated) string_literal, or nil if it has interpolation or isn't a string literal at all.



130
131
132
133
134
135
136
137
138
 
# File 'lib/scryer/ast.rb', line 130

def plain_string_value(node)
  return nil if string_literal_has_interpolation?(node)
  return nil unless tagged?(node, :string_literal)

  content = node[1]
  return nil unless tagged?(content, :string_content)

  content[1..].map { |part| part.is_a?(Array) && part[0] == :@tstring_content ? part[1] : nil }.compact.join
end

.position_of(node) ⇒ Object

Extract the [line, col] position from a node, searching its descendants for the first terminal token if the node itself isn't one. Returns nil if no position info can be found (shouldn't normally happen).



41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/scryer/ast.rb', line 41

def position_of(node)
  return nil unless node.is_a?(Array)

  # Terminal tokens look like [:@ident, "text", [line, col]]
  if node[0].is_a?(Symbol) && node[0].to_s.start_with?("@") && node[2].is_a?(Array) && node[2].size == 2
    return node[2]
  end

  node.each do |child|
    next unless child.is_a?(Array)

    pos = position_of(child)
    return pos if pos
  end

  nil
end

.references_params?(node) ⇒ Boolean

True if node is params, params[:x], or contains such a reference anywhere in its subtree — the shared "does this touch raw request data" check behind mass assignment, IDOR, SSRF, and path traversal detection. Says nothing about whether that reference is guarded (.permit, sanitization, an allowlist, ...) — callers that care about a specific guard (like MassAssignmentRule's .permit) check for it separately.

Returns:

  • (Boolean) 


305
306
307
308
309
310
311
 
# File 'lib/scryer/ast.rb', line 305

def references_params?(node)
  return false unless node.is_a?(Array)

  each_node(node).any? do |n|
    tagged?(n, :vcall, :var_ref, :fcall) && ident_text(n[1]) == "params"
  end
end

.source_line(source, line_number) ⇒ Object

Best-effort source-line snippet (1-indexed line number) for display in a finding.



141
142
143
144
145
 
# File 'lib/scryer/ast.rb', line 141

def source_line(source, line_number)
  return nil unless line_number

  source.lines[line_number - 1]&.strip
end

.source_text(source, node) ⇒ Object

Best-effort source text spanning every line node touches — used for duplicate-detection snippets (query chains, cache keys/values) where we want the literal source rather than a reconstructed one. Whole-line granularity: fine for a display snippet, but too coarse when the exact boundary matters (see exact_source_text below).



164
165
166
167
168
169
 
# File 'lib/scryer/ast.rb', line 164

def source_text(source, node)
  start_line, end_line = line_range_of(node)
  return nil unless start_line

  source.lines[(start_line - 1)...end_line]&.join
end

.string_literal_has_interpolation?(node) ⇒ Boolean

True if a [:string_literal, [:string_content, ...]] node contains any interpolation (:string_embexpr / :string_dvar children).

Returns:

  • (Boolean) 


122
123
124
125
126
 
# File 'lib/scryer/ast.rb', line 122

def string_literal_has_interpolation?(node)
  return false unless tagged?(node, :string_literal, :xstring_literal, :dyna_symbol)

  each_node(node).any? { |n| tagged?(n, :string_embexpr, :string_dvar) }
end

.tagged?(node, *tags) ⇒ Boolean

True if node is a tagged sexp node (e.g. [:def, ...]) whose tag is one of tags (symbols).

Returns:

  • (Boolean) 


34
35
36
 
# File 'lib/scryer/ast.rb', line 34

def tagged?(node, *tags)
  node.is_a?(Array) && node[0].is_a?(Symbol) && tags.include?(node[0])
end

.true_literal?(node) ⇒ Boolean

True if node is the literal true/false keyword ([:var_ref, [:@kw, "true"|"false", pos]]).

Returns:

  • (Boolean) 


285
286
287
 
# File 'lib/scryer/ast.rb', line 285

def true_literal?(node)
  kw_literal?(node) == "true"
end

.unwrap_args(node) ⇒ Object 



110
111
112
113
114
115
116
117
118
 
# File 'lib/scryer/ast.rb', line 110

def unwrap_args(node)
  return [] unless node.is_a?(Array)

  inner = tagged?(node, :arg_paren) ? node[1] : node
  return [] unless tagged?(inner, :args_add_block)

  list = inner[1]
  list.is_a?(Array) ? list : []
end