Class: SkillBench::Services::AgentSpawnerService

Inherits:
Object
  • Object
show all
Defined in:
lib/skill_bench/services/agent_spawner_service.rb

Overview

Spawns and executes LLM agents for evaluation.

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(evaluation, system_prompt, provider, config) ⇒ AgentSpawnerService

Returns a new instance of AgentSpawnerService.

Parameters:

  • evaluation (SkillBench::Models::Eval) —

    The eval being run

  • system_prompt (String) —

    The system prompt for the agent

  • provider (Object) —

    The resolved provider

  • config (Hash, nil) —

    Provider config



25
26
27
28
29
30
 
# File 'lib/skill_bench/services/agent_spawner_service.rb', line 25

def initialize(evaluation, system_prompt, provider, config)
  @evaluation = evaluation
  @system_prompt = system_prompt
  @provider = provider
  @config = config
end

Class Method Details

.call(evaluation, system_prompt, provider, config) ⇒ Hash

Spawns the LLM agent with the given system prompt.

Parameters:

  • evaluation (SkillBench::Models::Eval) —

    The eval being run

  • system_prompt (String) —

    The system prompt for the agent

  • provider (Object) —

    The resolved provider

  • config (Hash, nil) —

    Provider config

Returns:

  • (Hash) —

    Agent response with result, status, runtime, usage, raw_response, iterations



17
18
19
 
# File 'lib/skill_bench/services/agent_spawner_service.rb', line 17

def self.call(evaluation, system_prompt, provider, config)
  new(evaluation, system_prompt, provider, config).call
end

Instance Method Details

#call ⇒ Hash

Spawns the LLM agent with the given system prompt.

Returns:

  • (Hash) —

    Agent response with result, status, runtime, usage, raw_response, iterations



35
36
37
38
39
40
41
42
 
# File 'lib/skill_bench/services/agent_spawner_service.rb', line 35

def call
  return { result: 'mock result', status: :success, iterations: [] } if @provider.name == 'mock'

  client_params = build_client_params
  max_iterations = @config&.[](:max_iterations) || @config&.[]('max_iterations') || 25

  run_agent(client_params, max_iterations)
end