Class: Insika::SteerInjector

Inherits:
Object
  • Object
show all
Defined in:
lib/insika/steer_injector.rb

Overview

WHERE a message that arrived mid-run is allowed to enter the conversation.

A customer who corrects themselves while the agent is calling tools ("1234567", three seconds after "queria saber do pedido") should have that land before the model's next reasoning step, not after the whole run. RubyLLM runs the entire tool loop inside chat.ask, so the only place to append is from inside its callbacks — which is enough, because they are public and additive:

complete_once
├─ provider_completion         → assistant message announcing N tool_calls
├─ after_message(assistant)    ← N is read here
└─ handle_tool_calls
     ├─ add_tool_result_message ×N
     │    └─ after_message(tool)  ← counted; the Nth is THE BOUNDARY
     └─ halt_result || complete   ← the next model step sees what we appended

Counting to N is not an optimization, it is the correctness condition. A user message inserted BETWEEN tool results is rejected outright by Anthropic (all tool results of a batch must sit together) and merely tolerated by OpenAI.

Two invariants this object exists to keep:

· Tail-append only. Nothing already sent to the provider is edited, reordered or removed. That is what keeps the prompt cache valid, and a cache miss on a ~48k-token identity is a real cost, not a theoretical one. · A halted batch injects nothing. With halt_when there is no next model step (handle_tool_calls returns the Halt), so an appended message would sit in the transcript unanswered forever. The messages stay in the mailbox and the Executor releases them as a follow-up turn.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(chat:, actor:, policy:, emit:) ⇒ SteerInjector

chat: the turn's RubyLLM::Chat (already assembled). actor: the turn's TaskActor — the mailbox the steered messages arrive in. policy: the turn's QueuePolicy (frame decides how the text is worded). emit: ->(type, data) — the Executor's emitter, already bound to the task.



39
40
41
42
43
44
45
46
47
48
 
# File 'lib/insika/steer_injector.rb', line 39

def initialize(chat:, actor:, policy:, emit:)
  @chat = chat
  @actor = actor
  @policy = policy
  @emit = emit
  @expected = nil # tool calls announced by the batch in flight (nil = not in one)
  @seen = 0
  @halted = false
  @injected = 0
end

Instance Attribute Details

#injected ⇒ Object (readonly)

How many messages this run absorbed (read by specs and by the turn's event).



51
52
53
 
# File 'lib/insika/steer_injector.rb', line 51

def injected
  @injected
end

Instance Method Details

#message_ended(message) ⇒ Object

RubyLLM after_message. An assistant message carrying tool calls OPENS a batch; the Nth tool result CLOSES it, and that is the one boundary where appending is valid.



63
64
65
66
67
68
69
70
 
# File 'lib/insika/steer_injector.rb', line 63

def message_ended(message)
  role = field(message, :role).to_s
  return open_batch(message) if role == "assistant"
  return unless role == "tool" && @expected

  @seen += 1
  inject! if @seen >= @expected
end

#tool_result(result) ⇒ Object

RubyLLM after_tool_result, with the RAW result — the only place a Tool::Halt is still recognizable. By the time it becomes a role: tool message its content is the payload, indistinguishable from an ordinary result.



56
57
58
 
# File 'lib/insika/steer_injector.rb', line 56

def tool_result(result)
  @halted = true if halt?(result)
end