Class: Legion::Extensions::Llm::Mlx::Actor::MlxCallable

Inherits:
Object
  • Object
show all
Defined in:
lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb

Overview

Callable wrapper for an MLX provider instance. It is the exact-execution dispatch target: it implements the fleet dispatch operations (chat, stream_chat, embed, count_tokens) by delegating to a per-instance Mlx::Provider built from the instance config, plus the disconnect and normalize_dispatch_error(error:) contracts required by Inventory::CallableHandle and Routing::ProviderOutcome. Provider and Faraday errors are NOT rescued here so the coordinator's normalize_dispatch_error can classify them.

Constant Summary collapse

COMPLETION_NAMED_KEYS =

Keys the base Provider exposes as named kwargs for the completion operations. Anything else the fleet passes is folded into the payload params hash.

%i[tools temperature schema thinking tool_prefs headers].freeze
EMBED_NAMED_KEYS = 
%i[dimensions headers].freeze

Instance Method Summary collapse

Constructor Details

#initialize(instance_cfg:, logger:) ⇒ MlxCallable

Returns a new instance of MlxCallable.



738
739
740
741
742
743
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 738

def initialize(instance_cfg:, logger:)
  @instance_cfg = instance_cfg
  @logger = logger
  @disconnected = false
  @inference_calls = 0
end

Instance Method Details

#call_count ⇒ Object 



745
746
747
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 745

def call_count
  @inference_calls
end

#chat(messages:, model:, **rest) ⇒ Object

── Fleet dispatch operations ───────────────────────────────────



761
762
763
764
765
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 761

def chat(messages:, model:, **rest)
  record_inference
  named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
  provider.chat(messages: messages, model: model_info(model), params: params, **named)
end

#count_tokens(messages:, model:, **rest) ⇒ Object 



779
780
781
782
783
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 779

def count_tokens(messages:, model:, **rest)
  record_inference
  _named, params = split_fleet_kwargs(rest, [])
  provider.count_tokens(messages: messages, model: model, params: params)
end

#disconnect ⇒ Object 



753
754
755
756
757
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 753

def disconnect
  @disconnected = true
  @provider&.disconnect
  @logger.debug { '[mlx][callable] disconnected' }
end

#disconnected? ⇒ Boolean

Returns:

  • (Boolean) 


749
750
751
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 749

def disconnected?
  @disconnected
end

#embed(text:, model:, **rest) ⇒ Object 



773
774
775
776
777
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 773

def embed(text:, model:, **rest)
  record_inference
  named, params = split_fleet_kwargs(rest, EMBED_NAMED_KEYS)
  provider.embed(text: text, model: model_info(model), params: params, **named)
end

#normalize_dispatch_error(error:) ⇒ Object 



785
786
787
788
789
790
791
792
793
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 785

def normalize_dispatch_error(error:)
  reason = error.message.to_s[0, 512]
  kind = classify_error_kind(error: error)

  Legion::Extensions::Llm::Routing::ProviderOutcome.new(
    kind: kind,
    reason: reason.empty? ? 'unknown dispatch error' : reason
  )
end

#stream_chat(messages:, model:, **rest) ⇒ Object 



767
768
769
770
771
 
# File 'lib/legion/extensions/llm/mlx/actors/discovery_refresh.rb', line 767

def stream_chat(messages:, model:, **rest, &)
  record_inference
  named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
  provider.stream_chat(messages: messages, model: model_info(model), params: params, **named, &)
end