Class: Rigor::Analysis::IncrementalSession

Inherits:
Object
  • Object
show all
Defined in:
lib/rigor/analysis/incremental_session.rb

Overview

ADR-46 slice 2 — the in-memory incremental orchestrator that composes the recorded dependency graph (Runner#file_dependents), the affected closure (Rigor::Analysis::Incremental.affected), and the subset-analysis hook (Runner analyze_only:) into a working incremental re-check.

#baseline runs a full analysis with dependency recording and keeps, per analyzed file, its diagnostics (the cache), its content digest, and the per-file source set (to maintain the dependents index across rounds). #recheck digests the files again, computes the changed set ΔF, re-analyzes only ΔF ∪ dependents[ΔF], and serves every other analyzed file from the cache — the body tier.

The invariant the verify harness (and the spec) assert: #recheck's merged diagnostics are byte-identical (as a sorted set) to a full --no-cache re-analysis of the edited tree. This is the --verify-incremental acceptance gate, here without disk persistence or CLI wiring (the cache is in-process). It models the body tier only: an edit that adds / removes / moves a file is outside the analyzed set it maintains and falls to a fresh #baseline (the structural tier is a later slice). The class-length budget is relaxed: this is one cohesive orchestrator of the incremental state (per-file diagnostics cache, the file-level / symbol-level / negative dependency graphs, and the ADR-85 seed bundles), clearer read together than split across micro-classes that would all share the same ivars.

Defined Under Namespace

Classes: Recheck

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(configuration:, paths: nil, environment: nil, cache_store: nil, plugin_requirer: nil, workers: 0) ⇒ IncrementalSession

Returns a new instance of IncrementalSession.

Parameters:

  • paths (Array<String>, nil) (defaults to: nil) —

    explicit analysis roots; nil (the default) uses the configuration's paths:.

  • environment (Rigor::Environment, nil) (defaults to: nil) —

    optional shared environment to thread into each internal Runner. Long-lived callers and specs can use this to avoid rebuilding the same RBS universe for every baseline / recheck / oracle run.

  • cache_store (Rigor::Cache::Store, nil) (defaults to: nil) —

    ADR-85 WD1 — the persistent cache each internal Runner exposes to the RBS-env and plugin-producer tiers. A cross-process --incremental recheck otherwise rebuilt a fresh runner with no store, so every plugin #prepare producer (the ADR-9/#74/ADR-60 WD3 record-and-validate caches) recomputed per invocation — 86% of a Rails warm incremental. Threading the store lets those producers serve from disk. nil (the default) preserves the pre-#85 behaviour the specs assert; the whole-run ADR-45 result cache stays disabled on these runs (Runner#run_result_cacheable? excludes record_dependencies / analyze_only).

  • plugin_requirer (#call, nil) (defaults to: nil) —

    optional gem-require hook threaded into each internal Runner (mirrors Runner's parameter). nil (the default, and what the CLI passes) uses Kernel.require; embedders and specs inject a fake so a test plugin registers without touching the real load path.

  • workers (Integer) (defaults to: 0) —

    ADR-46 — the resolved fork-pool worker count threaded into every internal analyzer, so a --incremental recheck's closure re-analysis parallelises exactly like the standard check path (the recon's audit: --workers / RIGOR_RACTOR_WORKERS / parallel.workers: were silently ignored because build_runner passed no workers:). 0 (the default, and what the specs and --verify-incremental gate use) keeps the sequential recording path bit-for-bit unchanged. The fork pool records each worker's cross-file reads and marshals them back (PoolCoordinator), so the dependency graph a pooled recheck rebuilds equals the sequential one.



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/rigor/analysis/incremental_session.rb', line 64

def initialize(configuration:, paths: nil, environment: nil, cache_store: nil, plugin_requirer: nil,
               workers: 0)
  @configuration = configuration
  @paths = paths
  @environment = environment
  @cache_store = cache_store
  @plugin_requirer = plugin_requirer
  @workers = workers
  # ADR-85 WD2 — per-file discovery seed bundles keyed by logical path. A cold baseline builds them; a
  # warm recheck folds them (re-walking only changed files) and refreshes the set. Ride the snapshot.
  @seed_bundles = {}
  @cache = {}              # analyzed path => [Diagnostic]
  @sources = {}            # analyzed path => Set<source path it read from>
  @digests = {}            # analyzed path => ADR-87 packed stat-digest entry at last analysis
  @analyzed = []           # the project files analyzed last round
  @dependents = {}         # inverted @sources (file-level)
  # ADR-46 slice 4 — symbol-granularity tracking.
  @symbol_sources = {}     # consumer => { source_path => Set<"ClassName#method"> }
  @ancestry_sources = {}   # consumer => Set<source_path> (class-ancestry deps)
  @symbol_fingerprints = {}  # path => { "ClassName#method" => sha256_hex }
  @symbol_dependents = {}    # [source, symbol] => Set<consumer>
  @ancestry_dependents = {}  # source => Set<consumer> (inverted ancestry_sources)
  # ADR-46 slice 3 — negative (missing) dependencies: a consumer that
  # looked up a name and resolved nothing must be re-checked when that
  # name later appears (e.g. a `call.unresolved-toplevel` whose target
  # is defined by a later edit).
  @missing = {}              # consumer => Set<"kind:name"> it looked up and missed
  @negative_dependents = {}  # "kind:name" => Set<consumer> (inverted @missing)
  @class_decls = {}          # path => Set<qualified class name declared in the file>
  # ADR-89 WD2 — per-def observed-key return summaries: [path, symbol] => { keys:, returns:, effects: }.
  # Harvested from the ADR-84 return memo after each run; drives the behavioural-stability gate.
  @return_summaries = {}
  # ADR-88 WD1 — the plugin fact-surface digest computed for THIS invocation (nil until a
  # `#run_incremental` pass runs / a plugin-free project) and the reporting flags a caller (the CLI
  # banner + `--cache-stats`) reads after `#run_incremental`. `@last_runner` is the analysis runner the
  # post-hoc fingerprint reads from; `@plugin_fact_reusable` is the computed decision object.
  @plugin_fact_digest = nil
  @opaque_plugin_ids = [].freeze
  @fact_surface_invalidated = false
  @last_runner = nil
  @plugin_fact_reusable = nil
end

Instance Attribute Details

#opaque_plugin_ids ⇒ Object (readonly)

ADR-88 WD1 — reporting hooks the CLI reads after #run_incremental. fact_surface_invalidated? is true when a valid snapshot was dropped for a fact-surface reason (a plugin sig/catalog edit, or an opaque plugin) rather than a cold miss. opaque_plugin_ids names the contributing-but-surfaceless plugins that force a full run every invocation.



332
333
334
 
# File 'lib/rigor/analysis/incremental_session.rb', line 332

def opaque_plugin_ids
  @opaque_plugin_ids
end

Instance Method Details

#affected_closure(changed, added, removed) ⇒ Object

The frozen set of files a #recheck must re-analyse: the symbol/ancestry-granularity closure of the changed files (slice 4), the added files themselves, the consumers of any symbol / class that appeared in a changed OR added file (slice 3 — a now-defined call.unresolved-toplevel target or def.override-* ancestor), and the consumers of every removed file (which now miss what it provided). An added file has no before-state, so all its symbols / classes appear.



154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
 
# File 'lib/rigor/analysis/incremental_session.rb', line 154

def affected_closure(changed, added, removed)
  scan = changed + added
  # Parse the changed / added set ONCE for the per-symbol fingerprints, the class declarations, AND the
  # ADR-89 WD1 declaration signatures. They were separate `discovered_def_index_for_paths` passes over
  # the same `scan` set — a duplicate re-parse of every changed file each recheck (recon §2 / the P6
  # recheck-floor audit).
  summary = scan.empty? ? nil : Inference::ScopeIndexer.scan_summary_for_paths(scan)
  scan_index = summary && summary[:def_index]
  declaration_signatures = (summary && summary[:declaration_signatures]) || {}
  new_fps = symbol_fingerprints_from_index(scan_index)
  new_class_decls = class_declarations_from_index(scan_index)
  changed_pairs = Incremental.changed_symbol_pairs(changed, @symbol_fingerprints, new_fps)
  # ADR-89 WD1 — a changed file whose DECLARATION signature (per-def parameter shape / visibility /
  # ancestry / member layout / def line, bodies excluded) is byte-identical to the snapshot is
  # declaration-stable: every cross-file fact its ancestry / file-level dependents consume is
  # declaration-derived and therefore unchanged (a body edit is consumed only by SYMBOL dependents,
  # governed below by `changed_pairs`), so those dependents are skipped. Only the declaration-UNSTABLE
  # changed files contribute their ancestry / file-level dependents; the changed files themselves are
  # always re-analysed (`changed.to_set`) to learn their own diagnostics. This generalises the B1
  # comment-only gate (code-stable ⟹ declaration-stable) to same-line body edits.
  unstable = declaration_unstable(changed, declaration_signatures)
  # ADR-89 WD2 — a declaration-stable changed def whose behavioural surface (return type at every
  # previously-observed call key + content-mutation effects) is unchanged is behaviourally stable: its
  # symbol dependents' cached diagnostics stay valid, so drop them. `symbol_pairs` is `changed_pairs`
  # minus those stable pairs, and only it (not `changed_pairs`) drives the symbol-dependent fan-out.
  symbol_pairs = behaviourally_unstable_pairs(changed_pairs, unstable, scan_index)
  base = dependents_base(unstable, symbol_pairs)
  closure = base | changed.to_set | added.to_set | negative_affected(scan, new_fps, new_class_decls)
  removed.each { |path| closure |= @dependents[path] || Set.new }
  closure.freeze
end

#analyzed_files ⇒ Object

The project files analyzed at the last baseline / recheck — the set a verify pass partitions and the merge subtracts the affected closure from.



109
110
111
 
# File 'lib/rigor/analysis/incremental_session.rb', line 109

def analyzed_files
  @analyzed
end

#baseline ⇒ Object

Full baseline analysis with recording. Returns the run's diagnostics; populates the in-process cache

  • dependency state.


115
116
117
118
119
120
121
122
123
124
125
126
 
# File 'lib/rigor/analysis/incremental_session.rb', line 115

def baseline
  runner = build_runner(record_dependencies: true)
  diagnostics = run_runner(runner).diagnostics
  @last_runner = runner # ADR-88 WD1 — the post-hoc fact-surface fingerprint reads this prepared registry.
  @analyzed = runner.analyzed_files
  @seed_bundles = runner.seed_bundles # ADR-85 WD2 — the freshly built bundle set for the next run.
  absorb_dependency_graph(runner)
  @return_summaries = runner.return_summaries # ADR-89 WD2 — the full-run behavioural surface.
  @cache = per_file(diagnostics)
  @digests = @analyzed.to_h { |path| [path, pack_digest(path)] }
  diagnostics
end

#behaviourally_unstable_pairs(changed_pairs, unstable, scan_index) ⇒ Object

ADR-89 WD2 — changed_pairs minus the behaviourally-STABLE pairs whose symbol dependents may be skipped. A pair [path, "Class#method"] is a candidate when its file is declaration-stable (WD1), it carries a persisted return summary, and the (edited) def is GATE-ELIGIBLE — its only cross-file body surfaces are its return type and its content-mutation effects (no instance/class variable write, no yield, no implicit-self call whose transitive effect a caller could observe). For a candidate the session compares its content-mutation effects (pure AST) then re-evaluates its return at every persisted key (a runner probe); a pair passes only when BOTH are unchanged. Any missing summary, ineligible def, changed effect, changed return, or uncomputable key keeps the pair — the conservative direction the --verify-incremental and byte-identical recheck specs backstop.



206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
 
# File 'lib/rigor/analysis/incremental_session.rb', line 206

def behaviourally_unstable_pairs(changed_pairs, unstable, scan_index)
  return changed_pairs if @return_summaries.empty? || scan_index.nil?

  candidates = changed_pairs.select do |pair|
    !unstable.include?(pair.first) && @return_summaries.key?(pair) &&
      gate_eligible_def?(scan_def_node(pair.last, scan_index))
  end
  return changed_pairs if candidates.empty?

  effect_stable = candidates.select { |pair| effects_unchanged?(pair, scan_index) }
  return changed_pairs if effect_stable.empty?

  stable = return_stable_pairs(effect_stable, scan_index)
  stable.empty? ? changed_pairs : (changed_pairs - stable)
end

#comment_ingesting_plugin_loaded? ⇒ Boolean

Returns:

  • (Boolean) 


246
247
248
249
250
251
252
253
 
# File 'lib/rigor/analysis/incremental_session.rb', line 246

def comment_ingesting_plugin_loaded?
  # Mirrors the plugin loader's gem-name resolution (`ProjectPrePasses#trusted_gem_name`): a String
  # entry IS the gem name; a Hash entry names it under `"gem"` (or the manifest `"id"`).
  @configuration.plugins.any? do |entry|
    name = entry.is_a?(Hash) ? (entry["gem"] || entry["id"]) : entry
    COMMENT_INGESTING_PLUGIN_IDS.include?(name.to_s)
  end
end

#current_files ⇒ Object

The current project file set (cheap directory expansion, no analysis), used to detect files added / removed since the last run.



257
258
259
260
 
# File 'lib/rigor/analysis/incremental_session.rb', line 257

def current_files
  runner = build_runner
  @paths ? runner.analysis_file_set(@paths) : runner.analysis_file_set
end

#declaration_stable?(path, current_declaration_signature) ⇒ Boolean

Returns:

  • (Boolean) 


237
238
239
240
241
242
243
244
 
# File 'lib/rigor/analysis/incremental_session.rb', line 237

def declaration_stable?(path, current_declaration_signature)
  return false if current_declaration_signature.nil?

  bundle = @seed_bundles[path]
  return false if bundle.nil?

  bundle[:declaration_signature] == current_declaration_signature
end

#declaration_unstable(changed, declaration_signatures) ⇒ Object

ADR-89 WD1 — the changed files whose ancestry / file-level dependents must STILL be re-checked: those not provably declaration-stable. A changed file is declaration-STABLE (its ancestry / file-level dependents skippable) when its current ScopeIndexer.declaration_signature matches the one stored in the snapshot's seed bundle — i.e. its edit changed no method signature, visibility, ancestry, member layout, def line, or method existence, so every cross-file DECLARATION fact its dependents consume is unchanged. Falls back to treating EVERY changed file as unstable (today's full closure) when a comment-ingesting plugin is loaded — such a plugin reads the very comments the signature ignores, so a comment edit it treats as a no-op could change a cross-file type. Sorbet sigs / dry-types includes are CODE, captured by ADR-88's plugin-fact fingerprint (WD3), so only comment-as-input plugins escape here.



231
232
233
234
235
 
# File 'lib/rigor/analysis/incremental_session.rb', line 231

def declaration_unstable(changed, declaration_signatures)
  return changed if comment_ingesting_plugin_loaded?

  changed.reject { |path| declaration_stable?(path, declaration_signatures[path]) }
end

#dependents_base(unstable, symbol_pairs) ⇒ Object

The dependents contributed by the declaration-unstable changed files and the behaviourally-unstable symbol pairs: the ADR-46 slice-4 symbol-granular fan-out when either is present (ancestry deps of the unstable files + symbol deps of the changed pairs), else the coarse file-level fan-out.



189
190
191
192
193
194
195
 
# File 'lib/rigor/analysis/incremental_session.rb', line 189

def dependents_base(unstable, symbol_pairs)
  if symbol_pairs.any? || unstable.any? { |f| @ancestry_dependents[f] }
    Incremental.affected_with_symbols(unstable, symbol_pairs, @symbol_dependents, @ancestry_dependents)
  else
    Incremental.affected(unstable, @dependents)
  end
end

#fact_surface_invalidated? ⇒ Boolean

Returns:

  • (Boolean) 


334
335
336
 
# File 'lib/rigor/analysis/incremental_session.rb', line 334

def fact_surface_invalidated?
  @fact_surface_invalidated
end

#reanalyze_subset(subset) ⇒ Object

Verification engine (the --verify-incremental gate): with NO source edit, re-analyze subset fresh and serve every other analyzed file from the baseline cache. Because nothing on disk changed, the merged result MUST equal a full analysis — so this exercises the subset-analysis and cache-merge paths against a known-good oracle (a full --no-cache run) for an arbitrary partition, without mutating session state. Returns the merged diagnostics.



267
268
269
270
271
272
273
 
# File 'lib/rigor/analysis/incremental_session.rb', line 267

def reanalyze_subset(subset)
  affected = subset.to_set
  runner = build_runner(analyze_only: affected)
  fresh = run_runner(runner).diagnostics
  reused = @analyzed - affected.to_a
  fresh + reused.flat_map { |path| @cache[path] || [] }
end

#recheck ⇒ Object

Re-check after on-disk edits, including files added or removed since the last run (the structural tier). Re-analyzes only the affected closure and serves the rest from cache; refreshes the cache + dependency state so a subsequent #recheck sees the new world.



131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
 
# File 'lib/rigor/analysis/incremental_session.rb', line 131

def recheck
  previous = @analyzed
  current = current_files
  added = current - previous
  removed = previous - current
  changed = changed_paths(current & previous)
  affected = affected_closure(changed, added, removed)
  analyze_set = affected & current
  runner = build_runner(analyze_only: analyze_set, record_dependencies: true)
  fresh = run_runner(runner).diagnostics
  @last_runner = runner # ADR-88 WD1 — the post-hoc fact-surface fingerprint reads this prepared registry.
  reused = (current & previous) - affected.to_a
  merged = fresh + reused.flat_map { |path| @cache[path] || [] }
  absorb(runner, fresh, current, analyze_set, removed)
  Recheck.new(diagnostics: merged, changed: changed.to_set, added: added.to_set,
              removed: removed.to_set, affected: affected, reused: reused.to_set)
end

#run_incremental(snapshot:, fingerprint:) ⇒ Object

Cross-process incremental run (the --incremental flag's engine). With a disk snapshot whose fingerprint matches, restore the prior per-file state and #recheck (re-analyze only the changed closure, serve the rest from the restored cache); otherwise run a full #baseline. Either way, persist the updated snapshot for the next process. Returns [diagnostics, warm] — warm is true when a snapshot was restored. A nil fingerprint (uncomputable inputs) disables persistence: a plain full run.



281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
 
# File 'lib/rigor/analysis/incremental_session.rb', line 281

def run_incremental(snapshot:, fingerprint:)
  # ADR-87 WD1 — install the per-run digest table + recording instant + strict flag for the whole
  # invocation so change-detection's stat-then-digest freshness (`#pack_digest` / `#stat_fresh?`) honours
  # `cache.validation: digest` (and `RIGOR_STRICT_VALIDATION`, which the env-only path already sees) and
  # shares one digest memo across change detection and the baseline/absorb re-pack. The inner
  # `Runner#run` nests its own `with_run` for the analysis descriptors; nesting is safe (each restores).
  Cache::FileDigest.with_run(strict: @configuration.cache_validation_strict?) do
    restored = fingerprint && snapshot.load(fingerprint: fingerprint)
    # ADR-88 WD1 — the plugin fact-surface fingerprint gates snapshot reuse the same way the global
    # fingerprint gates the load: a plugin sig/catalog edit outside `signature_paths:` (a Sorbet `.rbi`)
    # changes the types unchanged call sites resolve, without moving any analyzed file, so the global
    # fingerprint stays fresh and the recheck would serve stale diagnostics. The fingerprint is computed
    # POST-HOC from the analysis runner (which already ran `#prepare` and validated its producers) so the
    # warm path pays no second `#prepare`; the decision is applied after the recheck. Opaque plugins
    # (types with no fingerprint surface) make the snapshot un-reusable.
    if restored
      restore(restored)
      result = recheck
      adopt_plugin_fact_fingerprint
      reuse = @plugin_fact_reusable.reusable_against?(restored.plugin_fact_digest)
      if reuse
        diagnostics = result.diagnostics
        warm = true
        # ADR-87 WD3 — a warm recheck that changed nothing leaves the session state byte-equivalent to the
        # snapshot it restored, so skip the unconditional rewrite (209ms + 2 MB on gitlab per null recheck).
        # A cold baseline always persists — there was no valid snapshot to reuse.
        skip_save = result.no_change?
      else
        # The fact surface moved (a plugin sig/catalog edit) or a plugin is opaque: the cached-served
        # files the recheck merged may be stale, so re-analyze the whole tree. The current fact-surface
        # digest (from the recheck runner) is unchanged by the re-analysis, so it is kept for the save.
        @fact_surface_invalidated = true
        diagnostics = baseline
        warm = false
        skip_save = false
      end
    else
      diagnostics = baseline
      adopt_plugin_fact_fingerprint
      warm = false
      skip_save = false
    end
    snapshot.save(fingerprint: fingerprint, payload: to_payload) if fingerprint && !skip_save
    [diagnostics, warm]
  end
end