Class: Agent::Sessions::Readers::Codex

Inherits:
Base
  • Object
show all
Defined in:
lib/agent/sessions/readers/codex.rb

Overview

Codex rollout files. Every mapping here was written against a real corpus rather than from the format notes: 415 files, 128,987 records, inventoried 2026-08-12. The distribution is the reason for several decisions below — response_item 57%, event_msg 38%, turn_context 4%, session_meta 416, world_state 182, inter_agent_communication_metadata 129, compacted 18.

Codex was chosen as the first reader for exactly this reason. pi was the planned reference implementation, but its store held no session files at all on the machine available, so every claim about its content would have been inference. A reference implementation has to be falsifiable.

Constant Summary collapse

NON_MESSAGE_TYPES =

Known, and deliberately not messages: the session header, per-turn configuration, and two state records Codex added in July 2026. Silence here is a judgement, not an oversight — these are not conversation, and warning about them would train a caller to ignore warnings.

%w[session_meta turn_context world_state
inter_agent_communication_metadata].freeze
ROLES =

"developer" is what Codex writes where the normalized vocabulary says :system. It is 101 of 292 role-bearing records in the sample, so this is the common path, not an edge case.

{ "user" => :user, "assistant" => :assistant, "developer" => :system,
"system" => :system, "tool" => :tool }.freeze
CONTENT_PARTS =

encrypted_content maps to :unknown deliberately, not for want of a better bucket: 80 real content items are encrypted by the model and this gem will never read them. Recognized-and-unreadable is a different thing from unrecognized, and only the second deserves a warning — a warning that fires on a permanent, understood condition is noise on every read.

{ "input_text" => :text, "output_text" => :text, "text" => :text,
"summary_text" => :text, "input_image" => :image,
"output_image" => :image, "encrypted_content" => :unknown }.freeze
TOOL_CALLS =

Every entry past the first three in each list came from running this reader over all 415 files and reading its own warnings: a 25-file sample showed none of them. Counts in that corpus: web_search_call 288, ghost_snapshot 197, agent_message 129, tool_search_call and tool_search_output 26 each, image_generation_call 1.

%w[custom_tool_call function_call local_shell_call
web_search_call tool_search_call].freeze
TOOL_OUTPUTS = 
%w[custom_tool_call_output function_call_output local_shell_call_output
tool_search_output].freeze
NON_MESSAGE_ITEMS =

Internal state that happens to travel as a response_item. Skipped in silence for the same reason turn_context is: it is not conversation, and a warning a caller must learn to ignore is worse than no warning.

%w[ghost_snapshot].freeze
CALL_INPUTS =

Where a tool call keeps what it was called with. custom_tool_call uses input, function_call uses arguments, web_search_call uses action, and tool_search_call uses arguments as a Hash rather than a String.

%w[input arguments action].freeze
CALL_OUTPUTS =

And where an output keeps its result.

%w[output tools].freeze

Constants inherited from Base

Base::MAX_RECORD_BYTES

Instance Attribute Summary

Attributes inherited from Base

#session

Instance Method Summary collapse

Methods inherited from Base

#branching?, #compactions, #each_message, #fidelity, #initialize, #messages, #partial?, #tree, #warnings

Constructor Details

This class inherits a constructor from Agent::Sessions::Readers::Base

Instance Method Details

#usage ⇒ Object

Session totals. Codex writes no usage on its messages; it writes token_count event records whose info.total_token_usage is a RUNNING TOTAL — verified against a real rollout on this machine (2026-08-24): consecutive records report total 33,751 then 69,135 while their last_token_usage differ, so the last record is the session and summing would multiply-count every earlier turn.

Two normalizations, both from that same file:

input_tokens INCLUDES cached_input_tokens (33,431 including 19,200
in the sample) — the opposite of Claude's disjoint spelling — so the
cached share is subtracted to make Usage#input mean one thing across
agents. Clamped at zero: a count that went negative would mean the
two fields disagree, and a wrong zero beats a negative token count.

cache_write_input_tokens maps to cache_creation. total_tokens is
deliberately not mapped anywhere: it restates the other fields, and
any bucket it landed in would be double-counted by a caller summing
buckets.


81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
 
# File 'lib/agent/sessions/readers/codex.rb', line 81

def usage
  info = nil
  each_record do |record, _line_number|
    next unless record["type"] == "event_msg"

    candidate = record.dig("payload", "info", "total_token_usage")
    info = candidate if record.dig("payload", "type") == "token_count" && candidate.is_a?(Hash)
  end
  return nil unless info

  input = count_from(info["input_tokens"])
  cached = count_from(info["cached_input_tokens"])
  mapped = Usage.new(input: input && cached ? [input - cached, 0].max : input,
                     output: count_from(info["output_tokens"]),
                     cache_read: cached,
                     cache_creation: count_from(info["cache_write_input_tokens"]),
                     reasoning: count_from(info["reasoning_output_tokens"]))
  # Same rule as Claude's usage_from: a token_count record whose every
  # field failed the count check answers nil, not an all-nil Usage.
  mapped.to_h.each_value.any? ? mapped : nil
end