Class: Brute::Middleware::ToolUseGuard

Inherits:
Object
  • Object
show all
Defined in:
lib/brute/middleware/tool_use_guard.rb

Overview

Guards against tool-only LLM responses where the assistant message is dropped from the context buffer.

When the LLM responds with only tool_use blocks (no text), llm.rb’s response adapter produces empty choices. The assistant message carrying tool_use blocks may be lost. This causes “unexpected tool_use_id” on the next call because tool_result references a tool_use that’s missing from the message history.

This middleware runs post-call and ensures every pending tool_use ID is covered by an assistant message in env. It handles three cases:

1. pending_functions is non-empty and the assistant message exists → no-op
2. pending_functions is non-empty but the assistant message is missing
   (or has different IDs) → inject synthetic message
3. pending_functions is empty (nil-choice bug) but the stream recorded
   tool calls → inject synthetic message using stream metadata

Instance Method Summary collapse

Constructor Details

#initialize(app) ⇒ ToolUseGuard

Returns a new instance of ToolUseGuard.



28
29
30
 
# File 'lib/brute/middleware/tool_use_guard.rb', line 28

def initialize(app)
  @app = app
end

Instance Method Details

#call(env) ⇒ Object 



32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
 
# File 'lib/brute/middleware/tool_use_guard.rb', line 32

def call(env)
  response = @app.call(env)

  # Collect pending tool data from env[:pending_functions] (primary)
  # or the stream's recorded metadata (fallback for nil-choice bug).
  tool_data = collect_tool_data(env)
  return response if tool_data.empty?

  # Find all tool_use IDs already covered by assistant messages.
  covered_ids = covered_tool_ids(env[:messages])

  # Inject a synthetic assistant message for any uncovered tool calls.
  uncovered = tool_data.reject { |td| covered_ids.include?(td[:id]) }
  inject_synthetic!(env[:messages], uncovered) unless uncovered.empty?

  response
end