Module: Pikuri::Os::Recoll
- Defined in:
- lib/pikuri/os/recoll.rb
Overview
Backend wrapper over the recoll full-text index, via its headless
recollq CLI (Xapian-backed). The DE-independent sibling of
LocalSearch: same seam shape (Recoll.search / Recoll.read_text /
Recoll.available? / Recoll.check_binaries!), interchangeable behind the
fileindex_search / fileindex_read tools, and Extension wires
whichever is available? first.
Recall on prose ties localsearch, so recoll is not a better
engine — it earns its place where localsearch is structurally blind
(source trees, .git checkouts, non-GNOME desktops). Its blind spot
is the inverse: recoll indexes exactly the folders the user
configured (+topdirs+), source code included — everything outside them
is invisible. The measured survey behind the pick-one-per-host order
is in pikuri-os/DESIGN.md.
Recoll.search asks recollq -F 'url abstract' for two base64-encoded
fields per line (recoll's own recommended machine shape) so paths and
snippets with spaces/brackets/newlines survive with no escaping
guesswork. Recoll.read_text uses recollq -d, which dumps a document's
stored/extracted text (PDFs/office docs included) after a header line.
Defined Under Namespace
Classes: CommandError
Constant Summary collapse
- BINARY =
Returns the headless recoll query CLI.
'recollq'- INDEXER =
Returns the recoll indexer/config CLI (used only to locate the config; never invoked to index from here).
'recollindex'- CONFIG_FILE =
Returns recoll config filename inside the config dir.
'recoll.conf'- DEFAULT_TOPDIRS =
recoll's compiled-in default when
topdirsis unset: the whole home directory (the sample config shipstopdirs = ~too). '~'- INDEX_DIR =
Returns the Xapian index directory name inside the config dir; its presence is what available? treats as "usable".
'xapiandb'
Class Method Summary collapse
-
.available? ⇒ Boolean
Non-raising usability probe — for callers that select a backend among several and want the one that can actually serve searches.
-
.check_binaries! ⇒ Object
Verify recollq is reachable; raise loudly otherwise.
-
.confdir ⇒ String
The recoll configuration directory recoll would use:
$RECOLL_CONFDIRif set and non-empty, else~/.recoll. -
.extract_dumped_text(raw, abs) ⇒ String?
Pull the dumped body of the result whose header line matches
absout ofrecollq -doutput. -
.indexed_dirs ⇒ Array<String>
Folders recoll is configured to index, read from
topdirsin the active config file. -
.label ⇒ String
Short human name for this backend, used by the fileindex tools when prefixing an
"Error: ..."observation. -
.limitations ⇒ String
Plain-English note of what this backend will not surface, appended to the fileindex tools' LLM description so the model knows when to fall back to a filesystem text/regex search.
-
.parse_search_output(raw) ⇒ Array<Hash>
Parse
recollq -F 'url abstract'output into hits. -
.parse_topdirs(raw) ⇒ Array<String>
Join backslash-continued lines, then read the
topdirsvalue from the global section (everything before the first[section]header). -
.read_text(path) ⇒ String?
Full stored/extracted text of an indexed file, via
recollq -dscoped to the file's own name. -
.search(query:, limit:) ⇒ Array<Hash>
Run
recollq -F 'url abstract' -n <limit>and parse the results.
Class Method Details
.available? ⇒ Boolean
Non-raising usability probe — for callers that select a backend
among several and want the one that can actually serve searches.
Stricter than check_binaries!: it requires both the recollq
binary and a built index (INDEX_DIR under confdir), because
an installed-but-never-indexed recoll errors ("Xapian index open
error") on every query — so it must not win backend selection over
a working alternative. See check_binaries! for why exit status is
ignored for the binary half.
210 211 212 213 214 215 216 217 |
# File 'lib/pikuri/os/recoll.rb', line 210 def self.available? return false unless File.directory?(File.join(confdir, INDEX_DIR)) Pikuri::Subprocess.spawn(BINARY, '-h', chdir: '/').wait true rescue Errno::ENOENT false end |
.check_binaries! ⇒ Object
Verify recollq is reachable; raise loudly otherwise. Presence is
judged by whether the binary ran (no Errno::ENOENT), not by exit
status: recollq -h exits non-zero by design, so a zero-exit probe
would false-negative on a perfectly good install.
194 195 196 197 198 |
# File 'lib/pikuri/os/recoll.rb', line 194 def self.check_binaries! Pikuri::Subprocess.spawn(BINARY, '-h', chdir: '/').wait rescue Errno::ENOENT raise install_hint end |
.confdir ⇒ String
The recoll configuration directory recoll would use: $RECOLL_CONFDIR
if set and non-empty, else ~/.recoll.
101 102 103 104 |
# File 'lib/pikuri/os/recoll.rb', line 101 def self.confdir env = ENV['RECOLL_CONFDIR'] env && !env.empty? ? File.(env) : File.('~/.recoll') end |
.extract_dumped_text(raw, abs) ⇒ String?
Pull the dumped body of the result whose header line matches abs
out of recollq -d output. Each result is a tab-delimited header
line (+
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 |
# File 'lib/pikuri/os/recoll.rb', line 171 def self.extract_dumped_text(raw, abs) want = "file://#{abs}" body = [] capturing = false raw.each_line do |line| url = line[%r{\t\[(file://[^\]]*)\]\t}, 1] if url break if capturing capturing = (url == want) elsif capturing body << line end end capturing || !body.empty? ? body.join.chomp : nil end |
.indexed_dirs ⇒ Array<String>
Folders recoll is configured to index, read from topdirs in the
active config file. Resolution mirrors recoll itself: the config dir
is $RECOLL_CONFDIR or ~/.recoll; topdirs is a space-separated
list (quoted entries may contain spaces, a trailing \ continues to
the next line) living in the global section, i.e. above the first
[section] header. Entries are +~+- and +$VAR+-expanded to absolute
paths. Defaults to the home dir (DEFAULT_TOPDIRS) when the config
is absent or names no topdirs, matching recoll's own default.
87 88 89 90 91 92 93 94 95 |
# File 'lib/pikuri/os/recoll.rb', line 87 def self.indexed_dirs path = File.join(confdir, CONFIG_FILE) raw = File.exist?(path) ? File.read(path) : nil entries = raw ? parse_topdirs(raw) : [DEFAULT_TOPDIRS] entries = [DEFAULT_TOPDIRS] if entries.empty? entries.map { |entry| (entry) } rescue SystemCallError [] end |
.label ⇒ String
Returns short human name for this backend, used by the
fileindex tools when prefixing an "Error: ..." observation.
54 |
# File 'lib/pikuri/os/recoll.rb', line 54 def self.label = 'recoll' |
.limitations ⇒ String
Plain-English note of what this backend will not surface, appended
to the fileindex tools' LLM description so the model knows when to
fall back to a filesystem text/regex search. A method, not a
constant like LocalSearch::LIMITATIONS, because the blind spot is
host-specific: the user's topdirs, only readable at runtime
(indexed_dirs). Falls back to a generic caveat when the config
can't be read.
65 66 67 68 69 70 71 72 73 74 |
# File 'lib/pikuri/os/recoll.rb', line 65 def self.limitations dirs = indexed_dirs return generic_limitation if dirs.empty? listed = dirs.map { |dir| "- #{dir}" }.join("\n") <<~LIMITS.chomp Only files under these folders are indexed (anything elsewhere on disk is not in this index — search those files' text directly instead): #{listed} LIMITS end |
.parse_search_output(raw) ⇒ Array<Hash>
Parse recollq -F 'url abstract' output into hits. Each data line is
two base64 tokens (url, abstract); recollq's "Recoll query: …" /
"N results" banner lines are skipped because their first token
isn't valid base64 for a file:// URL.
132 133 134 135 136 137 138 139 140 141 142 143 144 |
# File 'lib/pikuri/os/recoll.rb', line 132 def self.parse_search_output(raw) hits = [] raw.each_line do |line| url_b64, abstract_b64 = line.split(/\s+/, 2) url = decode_base64(url_b64) next unless url&.start_with?('file://') snippet = decode_base64(abstract_b64.to_s.strip) snippet = snippet&.strip hits << { path: url.delete_prefix('file://'), snippet: (snippet unless snippet.to_s.empty?) } end hits.uniq { |hit| hit[:path] } end |
.parse_topdirs(raw) ⇒ Array<String>
Join backslash-continued lines, then read the topdirs value from
the global section (everything before the first [section] header).
The last assignment wins (ConfSimple semantics).
225 226 227 228 229 230 231 232 233 234 235 236 |
# File 'lib/pikuri/os/recoll.rb', line 225 def self.parse_topdirs(raw) value = nil joined_lines(raw).each do |line| stripped = line.strip break if stripped.start_with?('[') # first [section] ends the global scope next if stripped.empty? || stripped.start_with?('#') key, _, rest = stripped.partition('=') value = rest if key.strip == 'topdirs' end value ? tokenize(value) : [] end |
.read_text(path) ⇒ String?
Full stored/extracted text of an indexed file, via recollq -d
scoped to the file's own name. Works for PDFs/office docs (extracted
at index time). nil when the path isn't in the index or has no
stored text — the caller turns that into an LLM-facing message.
153 154 155 156 157 158 159 160 161 |
# File 'lib/pikuri/os/recoll.rb', line 153 def self.read_text(path) abs = File.(path) result = Pikuri::Subprocess.spawn(BINARY, '-d', '-n', '50', "filename:#{File.basename(abs)}", chdir: '/').wait return nil unless result.status.success? text = extract_dumped_text(result.output, abs) text && !text.strip.empty? ? text : nil end |
.search(query:, limit:) ⇒ Array<Hash>
Run recollq -F 'url abstract' -n <limit> and parse the results.
113 114 115 116 117 118 119 120 121 122 123 |
# File 'lib/pikuri/os/recoll.rb', line 113 def self.search(query:, limit:) result = Pikuri::Subprocess.spawn(BINARY, '-F', 'url abstract', '-n', limit.to_s, query, chdir: '/').wait unless result.status.success? stderr = result.output.strip stderr = "exited #{result.status.exitstatus}" if stderr.empty? raise CommandError, stderr end parse_search_output(result.output) end |