Class: SkillBench::Services::AgentSpawnerService

Inherits:
Object
  • Object
show all
Defined in:
lib/skill_bench/services/agent_spawner_service.rb

Overview

Spawns and executes LLM agents for evaluation.

Constant Summary collapse

EMPTY_USAGE =

Zeroed token usage used when a run produces no usage data (e.g. mock, rescue).

{ prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }.freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(evaluation, system_prompt, provider, config) ⇒ AgentSpawnerService

Returns a new instance of AgentSpawnerService.

Parameters:

  • evaluation (SkillBench::Models::Eval) —

    The eval being run

  • system_prompt (String) —

    The system prompt for the agent

  • provider (Object) —

    The resolved provider

  • config (Hash, nil) —

    Provider config



28
29
30
31
32
33
 
# File 'lib/skill_bench/services/agent_spawner_service.rb', line 28

def initialize(evaluation, system_prompt, provider, config)
  @evaluation = evaluation
  @system_prompt = system_prompt
  @provider = provider
  @config = config
end

Class Method Details

.call(evaluation, system_prompt, provider, config) ⇒ Hash

Spawns the LLM agent with the given system prompt.

Parameters:

  • evaluation (SkillBench::Models::Eval) —

    The eval being run

  • system_prompt (String) —

    The system prompt for the agent

  • provider (Object) —

    The resolved provider

  • config (Hash, nil) —

    Provider config

Returns:

  • (Hash) —

    Agent response with result, status, runtime, usage, raw_response, iterations



20
21
22
 
# File 'lib/skill_bench/services/agent_spawner_service.rb', line 20

def self.call(evaluation, system_prompt, provider, config)
  new(evaluation, system_prompt, provider, config).call
end

Instance Method Details

#call ⇒ Hash

Spawns the LLM agent with the given system prompt.

Returns:

  • (Hash) —

    Agent response with result, status, runtime, usage, raw_response, iterations



38
39
40
41
42
43
44
45
 
# File 'lib/skill_bench/services/agent_spawner_service.rb', line 38

def call
  return { result: 'mock result', status: :success, iterations: [], usage: EMPTY_USAGE } if @provider.name == 'mock'

  client_params = build_client_params
  max_iterations = @config&.[](:max_iterations) || @config&.[]('max_iterations') || 25

  run_agent(client_params, max_iterations)
end