Module: Silas::Eval

Defined in:

lib/silas/eval.rb,
lib/silas/eval/dsl.rb,
lib/silas/eval/driver.rb,
lib/silas/eval/grader.rb,
lib/silas/eval/result.rb,
lib/silas/eval/runner.rb,
lib/silas/eval/assertions.rb,
lib/silas/eval/transcript.rb,
lib/silas/eval/scripted_engine.rb

Overview

Evals: fixtures are scenarios, assertions run against the DURABLE transcript (the real Session/Turn/Step/ToolInvocation rows). The harness only observes the exactly-once loop; it never touches it. Written like Rails tests, run as a deploy gate (non-zero exit on failure). The launch differentiator.

Defined Under Namespace

Modules: Assertions, Driver, Grader, Runner Classes: AssertionError, DSL, Result, Scenario, ScriptedEngine, Transcript

Class Method Summary

• .reset! ⇒ Object
• .scenario(name, tags: [], &block) ⇒ Object
• .scenarios ⇒ Object

Class Method Details

.reset! ⇒ Object

# File 'lib/silas/eval.rb', line 26

def reset!    = (@scenarios = [])

.scenario(name, tags: [], &block) ⇒ Object

# File 'lib/silas/eval.rb', line 28

def scenario(name, tags: [], &block)
  dsl = DSL.new(name, tags)
  dsl.instance_exec(&block)
  scenarios << dsl.to_scenario
end

.scenarios ⇒ Object

# File 'lib/silas/eval.rb', line 25

def scenarios = @scenarios ||= []