Class: CompletionKit::LlmClient

Inherits:
Object
  • Object
show all
Defined in:
app/services/completion_kit/llm_client.rb

Constant Summary collapse

DEFAULT_TEMPERATURE = 
0.7
REFUSAL_PHRASE =

Providers phrase a temperature refusal every which way. OpenAI's current reasoning models say "Unsupported value: 'temperature' does not support 0.7 with this model. Only the default (1) value is supported.", which shares no wording with Azure's "Unsupported parameter" or Anthropic's "not supported". Matching on any one phrasing silently turns a recoverable refusal into a failed row.

The wording can sit on either side of the parameter name, so both orders carry the same phrase set, and a provider that names the offending field outright gets a branch of its own.

Neither gap may cross a brace, which stops a refusal aimed at some other parameter from matching a temperature echoed back in a neighbouring object of the same body. The phrase-first gap additionally bars commas, because "unsupported parameter: logprobs. Supported: model, prompt, temperature" is a list of what IS allowed and must not read as a refusal. The temperature-first gap has to allow commas, since "temperature, top_p and top_k are not supported" is a genuine refusal.

/
  deprecated | not\s+supported | does\s+not\s+support |
  only\s+the\s+default | unsupported\s+(?:parameter|value)
/xi
TEMPERATURE_REFUSAL = 
/
  temperature [^{}]{0,80}? #{REFUSAL_PHRASE}
  |
  #{REFUSAL_PHRASE} [^{},]{0,80}? temperature
  |
  "param" \s*:\s* "temperature"
/xi

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config = {}) ⇒ LlmClient

Returns a new instance of LlmClient.



39
40
41
 
# File 'app/services/completion_kit/llm_client.rb', line 39

def initialize(config = {})
  @config = config
end

Class Method Details

.for_model(model_name, config = {}) ⇒ Object

Raises:

  • (ArgumentError) 


80
81
82
83
84
85
 
# File 'app/services/completion_kit/llm_client.rb', line 80

def self.for_model(model_name, config = {})
  provider = ApiConfig.provider_for_model(model_name)
  raise ArgumentError, "Unsupported model: #{model_name}" unless provider

  for_provider(provider, config)
end

.for_provider(provider_name, config = {}) ⇒ Object 



63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'app/services/completion_kit/llm_client.rb', line 63

def self.for_provider(provider_name, config = {})
  case provider_name.to_s
  when "openai"
    OpenAiClient.new(config)
  when "anthropic"
    AnthropicClient.new(config)
  when "ollama"
    OllamaClient.new(config)
  when "openrouter"
    OpenRouterClient.new(config)
  when "azure_foundry"
    AzureFoundryClient.new(config)
  else
    raise ArgumentError, "Unsupported provider: #{provider_name}"
  end
end

Instance Method Details

#available_models ⇒ Object

Raises:

  • (NotImplementedError) 


51
52
53
 
# File 'app/services/completion_kit/llm_client.rb', line 51

def available_models
  raise NotImplementedError, "Subclasses must implement available_models"
end

#configuration_errors ⇒ Object 



59
60
61
 
# File 'app/services/completion_kit/llm_client.rb', line 59

def configuration_errors
  []
end

#configured? ⇒ Boolean

Returns:

  • (Boolean)

Raises:

  • (NotImplementedError) 


55
56
57
 
# File 'app/services/completion_kit/llm_client.rb', line 55

def configured?
  raise NotImplementedError, "Subclasses must implement configured?"
end

#generate_completion(prompt, options = {}) ⇒ Object

Raises:

  • (NotImplementedError) 


47
48
49
 
# File 'app/services/completion_kit/llm_client.rb', line 47

def generate_completion(prompt, options = {})
  raise NotImplementedError, "Subclasses must implement generate_completion"
end

#temperature_dropped? ⇒ Boolean

Returns:

  • (Boolean) 


43
44
45
 
# File 'app/services/completion_kit/llm_client.rb', line 43

def temperature_dropped?
  @temperature_dropped == true
end