Class: SpreeMenuChat::LlmClient

Inherits:
Object
  • Object
show all
Defined in:
app/services/spree_menu_chat/llm_client.rb

Overview

Thin wrapper around the Gemini API's generateContent/streamGenerateContent REST endpoints. All generation calls in this extension go through here.

There is no official Google-maintained Ruby SDK for the Gemini API (confirmed — only unofficial community gems exist) — same "no SDK, call the REST API directly" situation as SpreeDoordash::Client. Auth is a plain API key on the query string (Gemini's own documented shape; unlike DoorDash there's no per-request signing).

Model default is the gemini-flash-lite-latest ALIAS, not a pinned dated model (see SpreeMenuChat::Configuration for why — confirmed live that a pinned model can 404 for new API keys while still appearing in the models.list catalog). If this ever 404s anyway, check the current model catalog and free-tier limits at https://aistudio.google.com.

Defined Under Namespace

Classes: MissingCredentialsError

Constant Summary collapse

BASE_URL = 
'https://generativelanguage.googleapis.com'.freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(credential: nil) ⇒ LlmClient

Returns a new instance of LlmClient.

Raises:



35
36
37
38
39
 
# File 'app/services/spree_menu_chat/llm_client.rb', line 35

def initialize(credential: nil)
  @credential = credential
  raise MissingCredentialsError, 'No Menu Chat credential connected for this store' unless @credential
  raise MissingCredentialsError, 'No Gemini API key set for this store' if @credential.gemini_api_key.blank?
end

Class Method Details

.for_store(store = Spree::Store.default) ⇒ Object 



31
32
33
 
# File 'app/services/spree_menu_chat/llm_client.rb', line 31

def self.for_store(store = Spree::Store.default)
  new(credential: SpreeMenuChat::Credential.find_by(store: store))
end

.instance ⇒ Object

Deliberately NOT memoized, same rationale as SpreeSquare::Client and SpreeDoordash::Client: a store's credential can be created/edited by an admin mid-process, and a Faraday connection is cheap enough to not be worth caching against that risk.



27
28
29
 
# File 'app/services/spree_menu_chat/llm_client.rb', line 27

def self.instance
  for_store
end

Instance Method Details

#generate(prompt, system_instruction: nil, model: SpreeMenuChat::Config.generation_model) ⇒ Object

Single-shot, non-streaming generation — used by M1's verify_connection rake task and SpreeMenuChat::AnswerGenerator#call. ChatController itself uses #generate_stream (below) as of M4; #call/#generate stay around as the simpler synchronous API for callers that don't need streaming (rake tasks, tests, a future admin "test this question" action).

system_instruction is Gemini's own top-level field for a system prompt (kept separate from contents, not concatenated into the user turn — this is the mechanism SpreeMenuChat::AnswerGenerator uses to scope the assistant to menu/FAQ-only, per the plan's read-only guardrail).



52
53
54
 
# File 'app/services/spree_menu_chat/llm_client.rb', line 52

def generate(prompt, system_instruction: nil, model: SpreeMenuChat::Config.generation_model)
  request(:post, "/v1beta/models/#{model}:generateContent", generate_body(prompt, system_instruction))
end

#generate_stream(prompt, system_instruction: nil, model: SpreeMenuChat::Config.generation_model, &block) ⇒ Object

Streaming counterpart of #generate (M4) — calls Gemini's own :streamGenerateContent?alt=sse endpoint and yields each text fragment to the block as it arrives, instead of waiting for the full response. Confirmed live (2026-08-16): Gemini's real SSE frames are data: <json> blocks separated by a blank line, using \r\n line endings (see #consume_sse_frames for why that specific detail is load-bearing), where each JSON object has the same candidates.content.parts.text shape #generate already parses — streaming just delivers it in more, smaller pieces, not a different shape. The very last frame typically carries an empty text plus a finishReason/thoughtSignature — skipped here since there's nothing to yield.

Uses Faraday's on_data streaming callback (confirmed live to work with this project's plain net_http adapter — no extra gem needed) rather than buffering the whole response and chunking it ourselves, so a slow/long answer actually reaches the caller incrementally instead of only feeling instant right at the very end.

Returns the final totalTokenCount Gemini reported (each SSE frame's usageMetadata carries a running total, so the last one seen is the true total for the whole turn) — SpreeMenuChat::AnswerGenerator feeds this straight into SpreeMenuChat::TokenBudget (M5) rather than estimating spend itself. Returns nil if the response never included usage data (shouldn't happen in practice, but callers must not assume a value).

Raises:



82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'app/services/spree_menu_chat/llm_client.rb', line 82

def generate_stream(prompt, system_instruction: nil, model: SpreeMenuChat::Config.generation_model, &block)
  raw = +''
  sse_buffer = +''
  total_tokens = nil

  response = connection.post("/v1beta/models/#{model}:streamGenerateContent") do |req|
    req.params['alt'] = 'sse'
    req.params['key'] = @credential.gemini_api_key
    req.headers['Content-Type'] = 'application/json'
    req.body = generate_body(prompt, system_instruction).to_json
    req.options.on_data = proc do |chunk, _bytes|
      raw << chunk
      sse_buffer << chunk
      seen = consume_sse_frames(sse_buffer, &block)
      total_tokens = seen if seen
    end
  end

  return total_tokens if response.status.between?(200, 299)

  # Confirmed live: an error response on this endpoint is plain JSON
  # (Gemini's normal {"error": {...}} shape), not an SSE "data:" frame
  # — it never matched the `data:` prefix above, so it's still sitting
  # in `raw` untouched here, same as #handle_response would see it in
  # response.body for the non-streaming endpoint.
  parsed = raw.present? ? JSON.parse(raw) : {}
  raise RequestError.new("Gemini API error (#{response.status}): #{parsed.inspect}", status: response.status, body: parsed)
end