Class: SkillBench::Tools::RunCommand

Inherits:
Object
  • Object
show all
Defined in:
lib/skill_bench/tools/run_command.rb

Overview

Handles executing a shell command within the working directory.

Real container isolation is not yet shipped, so an active sandbox means a temporary git directory on the host. To honor the documented security model the tool fails closed: when no container isolation is active it refuses to run unless allow_host_execution is explicitly enabled.

Constant Summary collapse

HOST_EXECUTION_REFUSED =

Refusal returned when no container isolation is active and host execution has not been explicitly enabled. Deliberately omits the allowlist.

'Command execution refused: no sandbox isolation is active and ' \
"'allow_host_execution' is not enabled. Set \"allow_host_execution\": true in " \
'skill-bench.json to permit running commands directly on the host (NOT isolated).'
HOST_EXECUTION_WARNING =

Warning emitted when a command runs un-isolated on the host because allow_host_execution is enabled and no container is active.

'Warning: running command directly on the host with NO sandbox isolation ' \
'(allow_host_execution is enabled). Commands are not isolated from your machine.'
TERM_GRACE_PERIOD =

Seconds to wait after SIGTERM before escalating to SIGKILL when a command exceeds its execution deadline.

2

Class Method Summary collapse

Class Method Details

.call(command, working_dir_path, container_id = nil) ⇒ String

Executes a shell command within the working directory (host or container).

Tokenizes the command string before execution so that arguments are passed directly to the OS without shell interpretation, preventing shell injection.

Fails closed: when no container isolation is active (container_id is nil) and allow_host_execution is false, the command is refused and nothing runs. When host execution is explicitly allowed, a warning is emitted once per command before running un-isolated on the host.

Parameters:

  • command (String) —

    The command to run (e.g. "rspec spec/models").

  • working_dir_path (Pathname) —

    The host directory (ignored if container_id present).

  • container_id (String, nil) (defaults to: nil) —

    The Docker container ID for isolated execution.

Returns:

  • (String) —

    A formatted string containing the exit status, STDOUT, and STDERR, or a standardized error/refusal message.



67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/skill_bench/tools/run_command.rb', line 67

def self.call(command, working_dir_path, container_id = nil)
  argv = command.shellsplit
  return 'Error: Empty command.' if argv.empty?

  base_cmd = argv.first
  return "Error: Command '#{base_cmd}' is blocked for security reasons." if Constants::Tools::DANGEROUS_COMMANDS.include?(base_cmd)

  allowed = SkillBench::Config.allowed_commands
  return 'Error: No allowed commands configured. Set allowed_commands in skill-bench.json or use --mode mock.' if allowed.nil?
  return "Error: Command '#{base_cmd}' is not permitted." unless allowed.include?(base_cmd)

  return "Error: Command '#{base_cmd}' arguments are not permitted by the configured argument constraints." unless arguments_permitted?(base_cmd, argv)

  return HOST_EXECUTION_REFUSED unless container_id || SkillBench::Config.allow_host_execution

  warn_unisolated_host_execution unless container_id
  execute(argv, working_dir_path, container_id)
end

.definition ⇒ Hash

Returns The tool definition for the LLM API.

Returns:

  • (Hash) —

    The tool definition for the LLM API.



34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/skill_bench/tools/run_command.rb', line 34

def self.definition
  {
    type: 'function',
    function: {
      name: 'run_command',
      description: 'Execute a shell command (e.g., rspec).',
      parameters: {
        type: 'object',
        properties: {
          command: { type: 'string', description: 'The shell command to run.' }
        },
        required: ['command'],
        additionalProperties: false
      }
    }
  }
end