Class: Insika::Evals::GraphTransport

Inherits:
Object
  • Object
show all
Defined in:
lib/insika/evals/transport.rb

Overview

A transport over the deployment's OWN graph, in-process — no HTTP. This is how a simulated conversation (or a replay) exercises the local agent the way a customer would reach it, tools and guardrails included, without a server in between. runtime is anything answering the DSL Runtime contract (#chat(message, session_id:, agent:) -> text, raising on failure) — the DSL Definition/System runtime, or a test double.

Tool activity is captured from the graph's event stream (the :tool_call events the ChatBuilder emits), so an in-process transcript records the same tool names an HTTP replay would — the Simulator's transcript is not blind to what the local agent called. event_stream is optional; when omitted it is read off the runtime's graph when one is reachable.

Instance Method Summary collapse

Constructor Details

#initialize(runtime:, event_stream: nil) ⇒ GraphTransport

Returns a new instance of GraphTransport.



192
193
194
195
196
 
# File 'lib/insika/evals/transport.rb', line 192

def initialize(runtime:, event_stream: nil)
  @runtime = runtime
  @event_stream = event_stream ||
                (runtime.graph.event_stream if runtime.respond_to?(:graph) && runtime.graph)
end

Instance Method Details

#mono ⇒ Object 



198
 
# File 'lib/insika/evals/transport.rb', line 198

def mono = Process.clock_gettime(Process::CLOCK_MONOTONIC)

#turn(agent:, conv:, message:) ⇒ Object 



200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
 
# File 'lib/insika/evals/transport.rb', line 200

def turn(agent:, conv:, message:)
  t0 = mono
  sub = @event_stream&.subscribe(types: [:tool_call])
  begin
    text = @runtime.chat(message, session_id: conv, agent: agent)
    TurnOutcome.new(
      result: TurnResult.new(output_text: text.to_s, tool_calls: drain_tools(sub), error: nil),
      ttfb: nil, total: (mono - t0) * 1000.0, usage: nil
    )
  rescue Insika::Error => e
    TurnOutcome.new(
      result: TurnResult.new(output_text: "", tool_calls: drain_tools(sub), error: e.message),
      ttfb: nil, total: (mono - t0) * 1000.0, usage: nil
    )
  ensure
    sub&.close
  end
end