Class: Esp::Mw::ReferenceIndex

Inherits:
Object
  • Object
show all
Defined in:
lib/esp/mw/reference_index.rb

Overview

SQLite index over the unpacked vanilla ESM JSONs. Replaces “grep over 200MB of JSON” with a millisecond keyed lookup.

Schema is intentionally narrow: just enough to answer “where is this record defined?” and “what’s its full JSON?”. Full-record retrieval falls back to reading the source ESM at the stored array index.

Storage location (step 23.5 slice 2): vanilla data is not per-project — it’s identical for every user, and ~150 MB of it. It lives at ‘$ESP_REFERENCES_DIR` (explicit override), else `$ESP_DATA_DIR/references/`, else `~/.config/esp/references/`. One index across every project. The defaults resolve at call time, not at require, so an env-var change between invocations takes effect.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(db_path: nil, source_dir: nil) ⇒ ReferenceIndex

Returns a new instance of ReferenceIndex.



36
37
38
39
 
# File 'lib/esp/mw/reference_index.rb', line 36

def initialize(db_path: nil, source_dir: nil)
  @source_dir = source_dir || self.class.default_source_dir
  @db_path = db_path || File.join(@source_dir, '.index.sqlite')
end

Instance Attribute Details

#db_path ⇒ Object (readonly)

Returns the value of attribute db_path.



34
35
36
 
# File 'lib/esp/mw/reference_index.rb', line 34

def db_path
  @db_path
end

#source_dir ⇒ Object (readonly)

Returns the value of attribute source_dir.



34
35
36
 
# File 'lib/esp/mw/reference_index.rb', line 34

def source_dir
  @source_dir
end

Class Method Details

.default_db_path ⇒ Object 



29
30
31
 
# File 'lib/esp/mw/reference_index.rb', line 29

def default_db_path
  File.join(default_source_dir, '.index.sqlite')
end

.default_source_dir ⇒ Object

Where vanilla ‘<Master>.esm.json` files live. Explicit override via $ESP_REFERENCES_DIR wins; otherwise fall under the per-user data dir (Esp::Recents.data_dir handles the ESP_DATA_DIR / MW_DATA_DIR / ~/.config/esp resolution).



25
26
27
 
# File 'lib/esp/mw/reference_index.rb', line 25

def default_source_dir
  ENV['ESP_REFERENCES_DIR'] || File.join(Esp::Recents.data_dir, 'references')
end

Instance Method Details

#count ⇒ Object 



58
59
60
 
# File 'lib/esp/mw/reference_index.rb', line 58

def count
  db.execute('SELECT COUNT(*) AS n FROM records').first['n']
end

#count_matching(query: nil, type: nil, like: nil, exact: false) ⇒ Object

Count of all rows matching the same filters (ignores limit), so callers can say “showing N of M”.



84
85
86
87
88
89
90
91
92
93
94
 
# File 'lib/esp/mw/reference_index.rb', line 84

def count_matching(query: nil, type: nil, like: nil, exact: false)
  clauses = []
  params = []
  apply_query(clauses, params, query, exact)
  apply_filter(clauses, params, 'id LIKE ?', like)
  apply_filter(clauses, params, 'type = ?', type)

  sql = +'SELECT COUNT(*) AS n FROM records'
  sql << " WHERE #{clauses.join(' AND ')}" unless clauses.empty?
  db.execute(sql, params).first['n']
end

#db ⇒ Object 



41
42
43
 
# File 'lib/esp/mw/reference_index.rb', line 41

def db
  @db ||= SQLite3::Database.new(@db_path).tap { |d| d.results_as_hash = true }
end

#fetch_record(source_esm, record_index) ⇒ Object

Pull the full JSON record for a hit by re-reading the source ESM.



97
98
99
 
# File 'lib/esp/mw/reference_index.rb', line 97

def fetch_record(source_esm, record_index)
  JSON.parse(File.read(File.join(@source_dir, "#{source_esm}.json")))[record_index]
end

#find(query: nil, type: nil, like: nil, exact: false, limit: 100) ⇒ Object

Search the index. By default ‘query` matches as a case-insensitive substring against BOTH id and name (so “Vivec” surfaces the god, the city’s cells, the region, books, dialogue, …), with exact-id hits sorted to the top. Pass exact: true for a precise id lookup. ‘like` is an explicit SQL LIKE pattern on id; `type` filters by record type. All filters AND together.



68
69
70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/esp/mw/reference_index.rb', line 68

def find(query: nil, type: nil, like: nil, exact: false, limit: 100)
  clauses = []
  params = []
  apply_query(clauses, params, query, exact)
  apply_filter(clauses, params, 'id LIKE ?', like)
  apply_filter(clauses, params, 'type = ?', type)

  sql = +'SELECT source_esm, record_index, type, id, name FROM records'
  sql << " WHERE #{clauses.join(' AND ')}" unless clauses.empty?
  sql << order_clause(query, exact, params)
  sql << " LIMIT #{limit.to_i}"
  db.execute(sql, params)
end

#rebuild! ⇒ Object 



45
46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/esp/mw/reference_index.rb', line 45

def rebuild!
  FileUtils.mkdir_p(@source_dir)
  FileUtils.rm_f(@db_path)
  @db = nil
  create_schema!
  sources = Dir["#{@source_dir}/*.esm.json"]
  raise Esp.t('errors.reference_index.no_sources', dir: @source_dir) if sources.empty?

  sources.sort.each { |path| index_esm(path) }
  db.execute('ANALYZE')
  count
end

#types_for(ids) ⇒ Object

Map a batch of ids → record type via one SQL query against the case-insensitive id index. Keys come back lowercased; the caller downcases its own lookups to match. Unknown ids are absent from the returned hash. Backs the cell-view marker colouring (step 21 slice 3).



105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/esp/mw/reference_index.rb', line 105

def types_for(ids)
  lowered = Array(ids).map { |id| id.to_s.downcase }.uniq
  return {} if lowered.empty?

  placeholders = (['?'] * lowered.size).join(', ')
  rows = db.execute(
    "SELECT LOWER(id) AS id, type FROM records WHERE LOWER(id) IN (#{placeholders})",
    lowered
  )
  # If duplicates exist across masters they're (in practice) the same
  # type; last-wins is fine.
  rows.to_h { |r| [r['id'], r['type']] }
end