Class: SmartPrompt::LlamacppAdapter

Inherits:
LLMAdapter show all
Defined in:
lib/smart_prompt/llm_adapter.rb

Instance Method Summary collapse

Constructor Details

#initialize(config) ⇒ LlamacppAdapter

Returns a new instance of LlamacppAdapter.



55
56
57
58
59
60
 
# File 'lib/smart_prompt/llm_adapter.rb', line 55

def initialize(config)
  super
  @client = OpenAI::Client.new(
    uri_base: @config['url']
  )
end

Instance Method Details

#send_request(messages, model = nil) ⇒ Object 



61
62
63
64
65
66
67
68
69
70
71
 
# File 'lib/smart_prompt/llm_adapter.rb', line 61

def send_request(messages, model=nil)
  SmartPrompt.logger.info "LlamacppAdapter: Sending request to Llamacpp"
  response = @client.chat(
    parameters: {
      messages: messages,
      temperature: @config['temperature'] || 0.7
    }
  )
  SmartPrompt.logger.info "LlamacppAdapter: Received response from Llamacpp"
  response.dig("choices", 0, "message", "content")
end