Class: Pikuri::VectorDb::Reranker::LlamaServer

Inherits:
Object
  • Object
show all
Defined in:
lib/pikuri/vector_db/reranker/llama_server.rb

Overview

Cross-encoder reranker via HTTP POST /v1/rerank against a llama.cpp server, in the same wire format Cohere's hosted reranker speaks — so a Reranker::Cohere later is a base-URL + auth-header swap on this client:

POST <endpoint>/v1/rerank   { "model"?, "query", "documents": [...] }
→ 200 { "results": [ { "index": 0, "relevance_score": 0.92 }, ... ] }

#rerank extracts results, re-sorts descending by relevance_score (defensively — servers usually pre-sort), and maps each to a Hit. endpoint: is the base URL (+/v1/rerank+ appended); most setups point it at the same router-mode llama-server as chat/embedder. model: is optional (llama.cpp infers the loaded model; Cohere requires it — set it forward-compatibly).

Errors are loud (same posture as Tokenizer::LlamaServer): non-2xx, missing results, malformed body, Faraday::Error all raise. The Tools::Search caller catches and falls back to vector-only top-k — outages degrade at the tool level, not by the client lying about scores.

Instance Method Summary collapse

Constructor Details

#initialize(endpoint:, model: nil, connection: nil) ⇒ LlamaServer

Parameters:

  • endpoint (String) —

    base URL of the rerank server, e.g. 'http://localhost:8082'.

  • model (String, nil) (defaults to: nil) —

    model name to send in the request body. nil omits the model field; llama.cpp uses whatever's loaded.

  • connection (Faraday::Connection, nil) (defaults to: nil) —

    optional dependency injection for tests.

Raises:

  • (ArgumentError) —

    on empty endpoint.



37
38
39
40
41
42
43
44
45
46
47
 
# File 'lib/pikuri/vector_db/reranker/llama_server.rb', line 37

def initialize(endpoint:, model: nil, connection: nil)
  raise ArgumentError, 'endpoint must be non-empty' if endpoint.nil? || endpoint.empty?

  @endpoint = endpoint
  @model = model
  @connection = connection || Faraday.new(url: endpoint) do |f|
    f.request :json
    f.response :json
    f.adapter Faraday.default_adapter
  end
end

Instance Method Details

#rerank(query:, documents:) ⇒ Array<Hit>

Score every document against query via the cross-encoder. Returns an Array<Hit> sorted descending by score; one entry per input document. Empty documents short-circuits to [].

Parameters:

  • query (String)
  • documents (Array<String>) —

    the candidates to score; positions become Hit.index values in the response.

Returns:

Raises:

  • (RuntimeError) —

    on HTTP non-2xx, missing results key, malformed entry, or any Faraday::Error (network failure, timeout).



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
 
# File 'lib/pikuri/vector_db/reranker/llama_server.rb', line 62

def rerank(query:, documents:)
  return [] if documents.empty?

  body = { query: query, documents: documents }
  body[:model] = @model if @model

  response = @connection.post('/v1/rerank') do |req|
    req.headers['Content-Type'] = 'application/json'
    req.body = body
  end

  unless response.status == 200
    raise "Reranker::LlamaServer: POST #{@endpoint}/v1/rerank returned " \
          "HTTP #{response.status}: #{response.body.inspect}"
  end

  results = response.body.is_a?(Hash) ? response.body['results'] : nil
  unless results.is_a?(Array)
    raise "Reranker::LlamaServer: response missing 'results' array " \
          "(got #{response.body.inspect})"
  end

  hits = results.map do |entry|
    idx = entry.is_a?(Hash) ? entry['index'] : nil
    score = entry.is_a?(Hash) ? entry['relevance_score'] : nil
    unless idx.is_a?(Integer) && score.is_a?(Numeric)
      raise "Reranker::LlamaServer: malformed result entry " \
            "(expected {index:Integer, relevance_score:Float}, got #{entry.inspect})"
    end

    Hit.new(index: idx, score: score.to_f)
  end
  hits.sort_by { |h| -h.score }
rescue Faraday::Error => e
  raise "Reranker::LlamaServer: #{e.class.name.split('::').last} " \
        "calling #{@endpoint}/v1/rerank: #{e.message}"
end