Module: Parse::Retrieval

Defined in:
lib/parse/retrieval/retriever.rb,
lib/parse/retrieval/chunk.rb,
lib/parse/retrieval/chunker.rb,
lib/parse/retrieval/reranker.rb,
lib/parse/retrieval/agent_tool.rb,
lib/parse/retrieval/reranker/cohere.rb

Overview

Retrieval-augmented-generation (RAG) helpers. Parse::RAG is a discoverability alias for this module.

Retrieval.retrieve is the agent-agnostic core: it embeds a natural-language query, runs Atlas $vectorSearch through the existing Class.find_similar (which enforces ACL/CLP mongo-direct), then splits each retrieved document's text field into scored Chunks for presentation.

The agent-facing semantic_search tool (see lib/parse/retrieval/agent_tool.rb) wraps Retrieval.retrieve with the agent security envelope (tenant scope, field_allowlist projection, score quantization).

== ACL model

Retrieval.retrieve does NOT implement a REST "two-stage" re-query. The vector path is mongo-direct only (Parse Server's REST /aggregate is master-key-only and bypasses ACL — see the project notes), and acl_user: / acl_role: scopes have no REST equivalent. ACL is enforced inside find_similar via a post-$vectorSearch _rperm $match. Scope kwargs (session_token: / acl_user: / acl_role: / master:) pass straight through **scope_opts.

Defined Under Namespace

Modules: AgentTool, Chunker, Reranker Classes: AmbiguousTextField, Chunk, TenantScopeConflict

Class Method Summary collapse

Class Method Details

.assert_no_underscore_keys!(obj, path = []) ⇒ Object

Recursively refuse any underscore-prefixed key, at any depth, in a caller-supplied filter. This is distinct from (and stricter than) the agent layer's flat validate_keys!: a Mongo-style filter is a nested structure, and an underscore key buried inside $or / $elemMatch / a hash value could clobber tenant scope or reach a reserved column (_rperm, _p_*, _auth_data_*). The walk is unconditional — it does not special-case operators.

Parameters:

  • obj (Object) —

    a filter Hash/Array (or anything; scalars pass).

  • path (Array<String>) (defaults to: []) —

    internal — accumulates the key path for the error message.

Raises:

  • (ArgumentError) —

    on any _-prefixed key.



56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
 
# File 'lib/parse/retrieval/retriever.rb', line 56

def assert_no_underscore_keys!(obj, path = [])
  case obj
  when Hash
    obj.each do |k, v|
      ks = k.to_s
      if ks.start_with?("_")
        raise ArgumentError,
              "filter key '#{(path + [ks]).join(".")}' is reserved (underscore-prefixed)."
      end
      assert_no_underscore_keys!(v, path + [ks])
    end
  when Array
    obj.each_with_index { |v, i| assert_no_underscore_keys!(v, path + ["[#{i}]"]) }
  end
  obj
end

.retrieve(query:, klass: nil, field: nil, text_field: nil, k: 10, filter: nil, vector_filter: nil, chunker: nil, tenant_scope: nil, score_quantize: false, source_transform: nil, hybrid: nil, rerank: nil, rerank_top_n: nil, **scope_opts) ⇒ Array<Parse::Retrieval::Chunk>

Retrieve and chunk documents semantically similar to query.

Parameters:

  • query (String) —

    natural-language query.

  • klass (Class, String) (defaults to: nil) —

    a Parse::Object subclass (or its class name) declaring a :vector property. class: is accepted as an alias.

  • field (Symbol, nil) (defaults to: nil) —

    the :vector property to search. Auto-resolved by find_similar when the class has exactly one.

  • text_field (Symbol, nil) (defaults to: nil) —

    the text property to chunk for presentation. Defaults to the sole text source of the class's embed declaration; raises AmbiguousTextField when it can't be inferred.

  • k (Integer) (defaults to: 10) —

    number of documents to retrieve. Default 10.

  • filter (Hash, nil) (defaults to: nil) —

    post-$vectorSearch $match filter.

  • vector_filter (Hash, nil) (defaults to: nil) —

    Atlas-native pre-search filter.

  • chunker (#chunk_with_meta, #chunk, nil) (defaults to: nil) —

    chunking strategy. Defaults to Parse::Retrieval::Chunker::FixedSizeOverlap.

  • tenant_scope (Hash, nil) (defaults to: nil) —

    { field:, value: } merged into vector_filter (closing the cross-tenant existence side channel) — not just a post-stage match.

  • score_quantize (Boolean) (defaults to: false) —

    round scores to 1 decimal (limits membership-inference probing in non-admin contexts).

  • source_transform (#call, nil) (defaults to: nil) —

    optional callable applied to each raw source record before it is stored on a Chunk. The agent tool injects tenant-scope assertion + field_allowlist projection here; a StandardError raised by the callable propagates and aborts the whole call (fail-closed). Kept as an injection point so this model-layer method stays free of any agent-layer dependency.

  • hybrid (Boolean, Hash, nil) (defaults to: nil) —

    when truthy, fuse a lexical Atlas Search branch with the $vectorSearch branch via reciprocal-rank fusion (see Core::VectorSearchable#hybrid_search). true uses defaults (lexical query = query); a Hash may carry :lexical, :vector, and :fusion sub-configs.

  • rerank (#rerank, nil) (defaults to: nil) —

    a Parse::Retrieval::Reranker::Base (or any object answering #rerank(query:, documents:, top_n:)). When present, retrieved documents are reordered by the cross-encoder relevance score BEFORE chunking, and the chunk score becomes the rerank relevance score.

  • rerank_top_n (Integer, nil) (defaults to: nil) —

    keep only the top-N documents after reranking (defaults to all retrieved documents).

  • scope_opts (Hash) —

    ACL/CLP scope kwargs forwarded verbatim to find_similar / hybrid_search: session_token: / acl_user: / acl_role: / master:.

Returns:



119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/parse/retrieval/retriever.rb', line 119

def retrieve(query:, klass: nil, field: nil, text_field: nil, k: 10,
             filter: nil, vector_filter: nil, chunker: nil,
             tenant_scope: nil, score_quantize: false,
             source_transform: nil, hybrid: nil, rerank: nil,
             rerank_top_n: nil, **scope_opts)
  if rerank && !rerank.respond_to?(:rerank)
    raise ArgumentError,
          "Parse::Retrieval.retrieve: `rerank:` must respond to #rerank " \
          "(a Parse::Retrieval::Reranker::Base); got #{rerank.class}."
  end

  # `class:` alias (reserved word — arrives via **scope_opts).
  klass ||= scope_opts.delete(:class)
  klass = resolve_class!(klass)

  unless query.is_a?(String) && !query.strip.empty?
    raise ArgumentError, "Parse::Retrieval.retrieve: `query:` must be a non-empty String."
  end

  resolved_text_field = (text_field || infer_text_field!(klass)).to_sym
  merged_vector_filter = fold_tenant_scope(klass, vector_filter, tenant_scope)
  chunker ||= default_chunker
  text_wire = wire_name(klass, resolved_text_field)

  raw_hits =
    if hybrid
      fetch_hybrid_hits(klass, query, k, field, filter, merged_vector_filter,
                        tenant_scope, hybrid, scope_opts)
    else
      klass.find_similar(
        text: query, k: k, field: field, filter: filter,
        vector_filter: merged_vector_filter, raw: true, **scope_opts,
      )
    end
  return [] if raw_hits.nil? || raw_hits.empty?

  raw_hits = apply_rerank(rerank, query, raw_hits, text_wire, rerank_top_n) if rerank

  raw_hits.flat_map do |doc|
    build_chunks_for(doc, klass, text_wire, score_quantize, source_transform, chunker)
  end
end