Class: Tryouts::CLI::AgentFormatter

Inherits:
Object
  • Object
show all
Includes:
FormatterInterface
Defined in:
lib/tryouts/cli/formatters/agent.rb

Overview

TOPAZ (Test Output Protocol for AI Zealots) Formatter

Language-agnostic test output format designed for LLM context management. This formatter implements the TOPAZ v1.0 specification for structured, token-efficient test result communication.

TOPAZ Features:

  • Language-agnostic field naming (snake_case, hierarchical)
  • Standardized execution context (runtime, environment, VCS)
  • Token budget awareness with smart truncation
  • Cross-platform compatibility (CI/CD, package managers)
  • Structured failure reporting with diffs
  • Protocol versioning for forward compatibility

Field Specifications:

  • command: Exact command executed
  • process_id: System process identifier
  • runtime: Language, version, platform info
  • package_manager: Dependency management system
  • version_control: VCS branch/commit info
  • environment: Normalized env vars (ci_system, app_env, etc.)
  • test_framework: Framework name, isolation mode, parser
  • execution_flags: Runtime flags in normalized form
  • protocol: TOPAZ version and configuration
  • project: Auto-detected project type
  • test_discovery: File pattern matching rules

Compatible with: Ruby/RSpec/Minitest, Python/pytest/unittest, JavaScript/Jest/Mocha, Java/JUnit, Go, C#/NUnit, and more.

Language Adaptation Examples: Python: runtime.language=python, package_manager.name=pip/poetry/conda Node.js: runtime.language=javascript, package_manager.name=npm/yarn/pnpm Java: runtime.language=java, package_manager.name=maven/gradle Go: runtime.language=go, package_manager.name=go_modules C#: runtime.language=csharp, package_manager.name=nuget/dotnet

Instance Attribute Summary

Attributes included from FormatterInterface

#current_indent, #stderr, #stdout

Instance Method Summary collapse

Methods included from FormatterInterface

#debug_info, #file_execution_start, #live_status_manager, #puts, #set_live_status_manager, #setup_output, #setup_start, #teardown_output, #teardown_start, #test_end, #test_output, #test_start, #trace_info, #update_live_status, #write

Constructor Details

#initialize(options = {}) ⇒ AgentFormatter

Returns a new instance of AgentFormatter.



48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 48

def initialize(options = {})
  super
  @budget = TokenBudget.new(options[:agent_limit] || TokenBudget::DEFAULT_LIMIT)
  @focus_mode = options[:agent_focus] || :failures
  @collected_files = []
  @current_file_data = nil
  @total_stats = { files: 0, tests: 0, failures: 0, errors: 0, elapsed_time: 0 }
  @output_rendered = false
  @options = options  # Store all options for execution context display
  @all_warnings = []  # Store warnings globally for execution details
  @syntax_errors = []  # Store syntax errors for execution details

  # No colors in agent mode for cleaner parsing
  @use_colors = false
end

Instance Method Details

#batch_summary(failure_collector) ⇒ Object

Summary operations - reliable trigger for rendering



171
172
173
174
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 171

def batch_summary(failure_collector)
  # NOTE: We don't render here because test_runner.rb calls grand_total directly
  # with accurate counts from the TestResultAggregator
end

#error_message(message, backtrace: nil) ⇒ Object 



195
196
197
198
199
200
201
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 195

def error_message(message, backtrace: nil)
  # Store syntax errors for display in execution details
  @syntax_errors << {
    message: message,
    backtrace: backtrace
  }
end

#file_end(file_path, context_info: {}) ⇒ Object 



84
85
86
87
88
89
90
91
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 84

def file_end(file_path, context_info: {})
  # Finalize current file data
  if @current_file_data
    @collected_files << @current_file_data
    @current_file_data = nil
  end
  # REMOVED: No longer attempts to render here to avoid premature output
end

#file_parsed(_file_path, test_count:, setup_present: false, teardown_present: false) ⇒ Object 



93
94
95
96
97
98
99
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 93

def file_parsed(_file_path, test_count:, setup_present: false, teardown_present: false)
  # Store per-file parsed test count for reference
  # Total executed tests come from TestResultAggregator via grand_total()
  if @current_file_data
    @current_file_data[:tests] = test_count
  end
end

#file_result(file_path, total_tests:, failed_count:, error_count:, elapsed_time: nil) ⇒ Object 



122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 122

def file_result(file_path, total_tests:, failed_count:, error_count:, elapsed_time: nil)
  # Update elapsed time (other stats come from TestResultAggregator via grand_total)
  @total_stats[:elapsed_time] += elapsed_time if elapsed_time

  # Update per-file data - file_result is called AFTER file_end, so data is in @collected_files
  relative_file_path = relative_path(file_path)
  file_data = @collected_files.find { |f| f[:path] == relative_file_path }

  if file_data
    file_data[:passed] = total_tests - failed_count - error_count
    # Also ensure tests count is correct if it wasn't set properly earlier
    file_data[:tests] ||= total_tests
  end
end

#file_start(file_path, context_info: {}) ⇒ Object

File-level operations - start collecting file data



73
74
75
76
77
78
79
80
81
82
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 73

def file_start(file_path, context_info: {})
  @current_file_data = {
    path: relative_path(file_path),
    tests: 0,
    failures: [],
    errors: [],
    passed: 0,
    context_info: context_info  # Store context info for later display
  }
end

#grand_total(total_tests:, failed_count:, error_count:, successful_files:, total_files:, elapsed_time:) ⇒ Object 



176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 176

def grand_total(total_tests:, failed_count:, error_count:, successful_files:, total_files:, elapsed_time:)
  return if @output_rendered  # Prevent double rendering

  # Use the accurate counts from TestResultAggregator (passed as parameters)
  # NOT the parsed counts we accumulated in file_parsed()
  @total_stats.merge!(
    tests: total_tests,           # Actual executed test count from aggregator
    failures: failed_count,       # Actual failures from aggregator
    errors: error_count,          # Actual errors from aggregator
    successful_files: successful_files,
    total_files: total_files,
    elapsed_time: elapsed_time,
  )

  # Now render all collected data
  render_agent_output
  @output_rendered = true
end

#live_status_capabilities ⇒ Object

Override live status - not needed for agent mode



204
205
206
207
208
209
210
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 204

def live_status_capabilities
  {
    supports_coordination: false,
    output_frequency: :none,
    requires_tty: false
  }
end

#parser_warnings(file_path, warnings:) ⇒ Object 



101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 101

def parser_warnings(file_path, warnings:)
  return if warnings.empty? || !@options.fetch(:warnings, true)

  # Store warnings globally for execution details and per-file
  warnings.each do |warning|
    warning_data = {
      type: warning.type.to_s,
      message: warning.message,
      line: warning.line_number,
      suggestion: warning.suggestion,
      file: relative_path(file_path)
    }
    @all_warnings << warning_data
  end

  # Also store in current file data for potential future use
  if @current_file_data
    @current_file_data[:warnings] = @all_warnings.select { |w| w[:file] == relative_path(file_path) }
  end
end

#phase_header(message, file_count: nil) ⇒ Object

Phase-level output - collect data, don't output immediately



65
66
67
68
69
70
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 65

def phase_header(message, file_count: nil)
  # Store file count for later use, but only store actual file count
  if file_count && message.include?("FILES")
    @total_stats[:files] = file_count
  end
end

#test_result(result_packet) ⇒ Object

Test-level operations - collect failure data



139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
 
# File 'lib/tryouts/cli/formatters/agent.rb', line 139

def test_result(result_packet)
  return unless @current_file_data

  # For summary mode, we still need to collect failures for counting, just don't build detailed data
  if result_packet.failed? || result_packet.error?
    if @focus_mode == :summary
      # Just track counts for summary
      if result_packet.error?
        @current_file_data[:errors] << { basic: true }
      else
        @current_file_data[:failures] << { basic: true }
      end
    else
      # Build detailed failure data for other modes
      failure_data = build_failure_data(result_packet)

      if result_packet.error?
        @current_file_data[:errors] << failure_data
      else
        @current_file_data[:failures] << failure_data
      end

      # Mark truncation for first-failure mode (handle limiting in render phase)
      if (@focus_mode == :first_failure || @focus_mode == :'first-failure') &&
         (@current_file_data[:failures].size + @current_file_data[:errors].size) > 1
        @current_file_data[:truncated] = true
      end
    end
  end
end