Class: Phronomy::TrustPipeline

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/trust_pipeline.rb

Overview

Orchestrates three trust mechanisms in a single pipeline:

  1. Citation Tracking — the DraftAgent is prompted to list the knowledge sources it relied on. Citations are extracted and attached to the result.

  2. Self-Review Loop — a dedicated ReviewAgent evaluates each draft, assigns a quality score, and provides actionable feedback. Rejected drafts are retried with the reviewer's feedback embedded in the next prompt.

  3. Confidence Gate — a combined confidence score (the minimum of the DraftAgent's self-reported confidence and the ReviewAgent's score) is compared against a threshold. The pipeline finishes early when the gate passes; after +max_iterations+ cycles it finishes regardless and marks the result as untrusted when the threshold was not reached.

Examples:

pipeline = Phronomy::TrustPipeline.new(
  draft_agent:          PolicyDraftAgent,
  review_agent:         PolicyReviewAgent,
  confidence_threshold: 0.7,
  max_iterations:       3
)
result = pipeline.invoke("What is the refund policy?")
puts result.output      # the final answer string
puts result.trusted?    # true when confidence >= threshold
result.citations.each { |c| puts "#{c[:source]}: #{c[:excerpt]}" }

Defined Under Namespace

Classes: Result

Constant Summary collapse

DEFAULT_CONFIDENCE_THRESHOLD =

Default confidence threshold for trusting an answer.

0.7
DEFAULT_MAX_ITERATIONS =

Default maximum draft-review cycles before returning best effort.

3

Instance Method Summary collapse

Constructor Details

#initialize(draft_agent:, review_agent:, confidence_threshold: DEFAULT_CONFIDENCE_THRESHOLD, max_iterations: DEFAULT_MAX_ITERATIONS) ⇒ TrustPipeline

Returns a new instance of TrustPipeline.

Parameters:

  • draft_agent (Class) —

    subclass of Phronomy::Agent::Base

  • review_agent (Class) —

    subclass of Phronomy::Agent::Base

  • confidence_threshold (Float) (defaults to: DEFAULT_CONFIDENCE_THRESHOLD) —

    answers below this are retried (default: 0.7)

  • max_iterations (Integer) (defaults to: DEFAULT_MAX_ITERATIONS) —

    maximum draft-review cycles (default: 3)



78
79
80
81
82
83
84
85
86
87
 
# File 'lib/phronomy/trust_pipeline.rb', line 78

def initialize(draft_agent:, review_agent:,
  confidence_threshold: DEFAULT_CONFIDENCE_THRESHOLD,
  max_iterations: DEFAULT_MAX_ITERATIONS)
  @draft_agent_class = draft_agent
  @review_agent_class = review_agent
  @threshold = confidence_threshold.to_f
  @max_iterations = max_iterations.to_i
  @graph_mutex = Mutex.new
  @compiled_graph = nil
end

Instance Method Details

#invoke(input, config: {}) ⇒ Result

Run the pipeline.

Parameters:

  • input (String) —

    the user question or task description

  • config (Hash) (defaults to: {}) —

    forwarded to the underlying agents (e.g. thread_id)

Returns:



94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/phronomy/trust_pipeline.rb', line 94

def invoke(input, config: {})
  app = compiled_graph
  state = app.invoke({input: input}, config: config)
  confidence = combined_confidence(state)
  Result.new(
    output: state.output || state.draft.to_s,
    confidence: confidence,
    citations: state.citations,
    iterations: state.iteration,
    review_notes: state.review_notes,
    trusted: confidence >= @threshold
  )
end