Class: SoilSidekick::TurboQuantApi

Inherits:
Object
  • Object
show all
Defined in:
lib/soilsidekick/api/turbo_quant_api.rb

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(api_client = ApiClient.default) ⇒ TurboQuantApi

Returns a new instance of TurboQuantApi.



19
20
21
 
# File 'lib/soilsidekick/api/turbo_quant_api.rb', line 19

def initialize(api_client = ApiClient.default)
  @api_client = api_client
end

Instance Attribute Details

#api_client ⇒ Object

Returns the value of attribute api_client.



17
18
19
 
# File 'lib/soilsidekick/api/turbo_quant_api.rb', line 17

def api_client
  @api_client
end

Instance Method Details

#turbo_quant_capabilities(turbo_quant_capabilities_request, opts = {}) ⇒ TurboQuantCapabilities

Query TurboQuant device capabilities Returns hardware-specific model recommendations and estimated performance based on detected device capabilities. Use this to determine the optimal local inference configuration for the end-user’s device. **Tier requirement:** Pro or Enterprise

Parameters:

Options Hash (opts):

  • :x_tq_context_mode (Integer) —

    TurboQuant extended context window size (tokens). With 3-bit KV cache, context windows up to 24K tokens are feasible within the same memory budget as standard 4K. (default to 4096)

  • :x_tq_kv_cache_hint (String) —

    KV cache management hint. - `none`: No caching (default) - `reuse`: Reuse KV cache from previous request in same session (40-60% compute savings) - `persist`: Persist cache to disk for cross-session reuse (default to ‘none’)

  • :x_tq_model_tier (String) —

    Preferred model tier for inference. - `auto`: Server selects optimal model based on device capabilities - `gemma-2b`: Lightweight model (~0.5GB KV with TQ) - `gemma-7b`: Full model, standard KV cache - `gemma-7b-tq`: Full model with TurboQuant 3-bit KV cache (~1.3GB) (default to ‘auto’)

Returns:



30
31
32
33
 
# File 'lib/soilsidekick/api/turbo_quant_api.rb', line 30

def turbo_quant_capabilities(turbo_quant_capabilities_request, opts = {})
  data, _status_code, _headers = turbo_quant_capabilities_with_http_info(turbo_quant_capabilities_request, opts)
  data
end

#turbo_quant_capabilities_with_http_info(turbo_quant_capabilities_request, opts = {}) ⇒ Array<(TurboQuantCapabilities, Integer, Hash)>

Query TurboQuant device capabilities Returns hardware-specific model recommendations and estimated performance based on detected device capabilities. Use this to determine the optimal local inference configuration for the end-user&#39;s device. **Tier requirement:** Pro or Enterprise

Parameters:

Options Hash (opts):

  • :x_tq_context_mode (Integer) —

    TurboQuant extended context window size (tokens). With 3-bit KV cache, context windows up to 24K tokens are feasible within the same memory budget as standard 4K. (default to 4096)

  • :x_tq_kv_cache_hint (String) —

    KV cache management hint. - &#x60;none&#x60;: No caching (default) - &#x60;reuse&#x60;: Reuse KV cache from previous request in same session (40-60% compute savings) - &#x60;persist&#x60;: Persist cache to disk for cross-session reuse (default to ‘none’)

  • :x_tq_model_tier (String) —

    Preferred model tier for inference. - &#x60;auto&#x60;: Server selects optimal model based on device capabilities - &#x60;gemma-2b&#x60;: Lightweight model (~0.5GB KV with TQ) - &#x60;gemma-7b&#x60;: Full model, standard KV cache - &#x60;gemma-7b-tq&#x60;: Full model with TurboQuant 3-bit KV cache (~1.3GB) (default to ‘auto’)

Returns:

  • (Array<(TurboQuantCapabilities, Integer, Hash)>) —

    TurboQuantCapabilities data, response status code and response headers



43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/soilsidekick/api/turbo_quant_api.rb', line 43

def turbo_quant_capabilities_with_http_info(turbo_quant_capabilities_request, opts = {})
  if @api_client.config.debugging
    @api_client.config.logger.debug 'Calling API: TurboQuantApi.turbo_quant_capabilities ...'
  end
  # verify the required parameter 'turbo_quant_capabilities_request' is set
  if @api_client.config.client_side_validation && turbo_quant_capabilities_request.nil?
    fail ArgumentError, "Missing the required parameter 'turbo_quant_capabilities_request' when calling TurboQuantApi.turbo_quant_capabilities"
  end
  allowable_values = [4096, 8192, 16384, 24576]
  if @api_client.config.client_side_validation && opts[:'x_tq_context_mode'] && !allowable_values.include?(opts[:'x_tq_context_mode'])
    fail ArgumentError, "invalid value for \"x_tq_context_mode\", must be one of #{allowable_values}"
  end
  allowable_values = ["none", "reuse", "persist"]
  if @api_client.config.client_side_validation && opts[:'x_tq_kv_cache_hint'] && !allowable_values.include?(opts[:'x_tq_kv_cache_hint'])
    fail ArgumentError, "invalid value for \"x_tq_kv_cache_hint\", must be one of #{allowable_values}"
  end
  allowable_values = ["auto", "gemma-2b", "gemma-7b", "gemma-7b-tq"]
  if @api_client.config.client_side_validation && opts[:'x_tq_model_tier'] && !allowable_values.include?(opts[:'x_tq_model_tier'])
    fail ArgumentError, "invalid value for \"x_tq_model_tier\", must be one of #{allowable_values}"
  end
  # resource path
  local_var_path = '/turbo-quant-capabilities'

  # query parameters
  query_params = opts[:query_params] || {}

  # header parameters
  header_params = opts[:header_params] || {}
  # HTTP header 'Accept' (if needed)
  header_params['Accept'] = @api_client.select_header_accept(['application/json']) unless header_params['Accept']
  # HTTP header 'Content-Type'
  content_type = @api_client.select_header_content_type(['application/json'])
  if !content_type.nil?
      header_params['Content-Type'] = content_type
  end
  header_params[:'x-tq-context-mode'] = opts[:'x_tq_context_mode'] if !opts[:'x_tq_context_mode'].nil?
  header_params[:'x-tq-kv-cache-hint'] = opts[:'x_tq_kv_cache_hint'] if !opts[:'x_tq_kv_cache_hint'].nil?
  header_params[:'x-tq-model-tier'] = opts[:'x_tq_model_tier'] if !opts[:'x_tq_model_tier'].nil?

  # form parameters
  form_params = opts[:form_params] || {}

  # http body (model)
  post_body = opts[:debug_body] || @api_client.object_to_http_body(turbo_quant_capabilities_request)

  # return_type
  return_type = opts[:debug_return_type] || 'TurboQuantCapabilities'

  # auth_names
  auth_names = opts[:debug_auth_names] || ['ApiKeyAuth']

  new_options = opts.merge(
    :operation => :"TurboQuantApi.turbo_quant_capabilities",
    :header_params => header_params,
    :query_params => query_params,
    :form_params => form_params,
    :body => post_body,
    :auth_names => auth_names,
    :return_type => return_type
  )

  data, status_code, headers = @api_client.call_api(:POST, local_var_path, new_options)
  if @api_client.config.debugging
    @api_client.config.logger.debug "API called: TurboQuantApi#turbo_quant_capabilities\nData: #{data.inspect}\nStatus code: #{status_code}\nHeaders: #{headers}"
  end
  return data, status_code, headers
end