Class: Scryer::Scanner

Inherits:
Object
  • Object
show all
Defined in:
lib/scryer/scanner.rb

Overview

Ties parsing + rules + duplicate detection together across a directory tree. This is the one entry point both the sentinel:scan rake task and any generator/CLI should call — everything else in this gem is a building block this class composes.

Defined Under Namespace

Classes: Result

Constant Summary collapse

DEFAULT_GLOB_DIRS = 
%w[app lib config db].freeze
SKIP_DIR_SEGMENTS = 
%w[vendor node_modules tmp log .git spec test].freeze
DUPLICATE_DETECTION_APP_SUBDIRS =

Duplicate-code detection only makes sense within hand-written business logic. Migrations in particular are mostly boilerplate (create_table / t.string / t.integer calls) that normalize to near-identical token streams and drown out real duplicates with false positives — so unlike the security/performance rules (which still scan every file under DEFAULT_GLOB_DIRS), duplicate detection is restricted to models, controllers, helpers, and concerns (wherever they're nested).

%w[models controllers helpers].freeze
QUERY_SIMILARITY_THRESHOLD =

Query and cache-value duplicates use a slightly higher bar than whole methods: they're much shorter fragments, so a coincidental match at the method threshold (0.6) is more likely — see DuplicateDetector's shingle comment for why short token streams are more sensitive to this.

0.7
CACHE_SIMILARITY_THRESHOLD = 
0.7

Instance Method Summary collapse

Constructor Details

#initialize(root:, dirs: DEFAULT_GLOB_DIRS, skip_rules: [], detect_duplicates: true) ⇒ Scanner

skip_rules silences specific checks by rule_id (e.g. a known false positive on this codebase) without editing/removing the rule itself — accepts strings or symbols, matched against Rule.rule_id.

detect_duplicates: false skips duplicate-code detection entirely (method/query/cache-key extraction and the DuplicateDetector passes below) — unlike the security/performance/style rules, duplicate detection isn't a Scryer::Rule with its own rule_id, so skip_rules has no way to address it; this is its equivalent off switch. See Scryer::Configuration#detect_duplicates for the config-driven default every CLI/rake entry point reads before constructing a Scanner.



42
43
44
45
46
47
 
# File 'lib/scryer/scanner.rb', line 42

def initialize(root:, dirs: DEFAULT_GLOB_DIRS, skip_rules: [], detect_duplicates: true)
  @root = File.expand_path(root)
  @dirs = dirs
  @skip_rules = Set.new(skip_rules.map(&:to_s))
  @detect_duplicates = detect_duplicates
end

Instance Method Details

#call ⇒ Object 



49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/scryer/scanner.rb', line 49

def call
  files = collect_files
  all_methods = []
  all_queries = []
  all_cache_calls = []
  security_findings = []
  performance_findings = []
  style_findings = []
  parse_errors = []

  files.each do |abs_path|
    rel_path = abs_path.sub(/\A#{Regexp.escape(@root)}\/?/, "")
    source = File.read(abs_path)

    sexp = begin
      Ripper.sexp(source)
    rescue StandardError => e
      parse_errors << { file: rel_path, error: e.message }
      nil
    end

    if sexp.nil?
      parse_errors << { file: rel_path, error: "Ripper could not parse this file (possibly a syntax error, or Ruby syntax newer than this gem's Ruby runtime supports)" } unless parse_errors.any? { |pe| pe[:file] == rel_path }
      next
    end

    RuleSet.all.each do |rule_class|
      next if @skip_rules.include?(rule_class.rule_id.to_s)

      bucket =
        case rule_class.category
        when "security" then security_findings
        when "performance" then performance_findings
        when "style" then style_findings
        end
      next unless bucket

      bucket.concat(rule_class.new(file: rel_path, source: source, sexp: sexp).scan)
    end

    if @detect_duplicates && duplicate_detection_target?(rel_path)
      all_methods.concat(MethodExtractor.extract(file: rel_path, source: source, sexp: sexp))
      all_queries.concat(QueryExtractor.extract(file: rel_path, source: source, sexp: sexp))
      all_cache_calls.concat(CacheExtractor.extract(file: rel_path, source: source, sexp: sexp))
    end
  end

  duplicate_groups =
    if @detect_duplicates
      # Same computed value cached under the same key from multiple call
      # sites is normal (just reusing the cache). Only flag it when the
      # *keys* differ too — that's either a redundant cache entry or a
      # key that drifted out of sync with a copy-pasted sibling.
      cache_groups = DuplicateDetector.call(all_cache_calls, threshold: CACHE_SIMILARITY_THRESHOLD, kind: "cache_duplicate")
                                       .select { |g| g.members.map(&:cache_key).uniq.size > 1 }

      DuplicateDetector.call(all_methods, kind: "method_duplicate") +
        DuplicateDetector.call(all_queries, threshold: QUERY_SIMILARITY_THRESHOLD, kind: "query_duplicate") +
        cache_groups
    else
      []
    end

  Result.new(
    security_findings: security_findings,
    performance_findings: performance_findings,
    style_findings: style_findings,
    duplicate_groups: duplicate_groups,
    files_scanned: files.size,
    parse_errors: parse_errors
  )
end