Class: Roast::Cogs::Chat

Inherits:
Roast::Cog show all
Defined in:
lib/roast/cogs/chat.rb,
lib/roast/cogs/chat/input.rb,
lib/roast/cogs/chat/config.rb,
lib/roast/cogs/chat/output.rb,
lib/roast/cogs/chat/session.rb

Overview

Chat cog for pure LLM interaction

The chat cog provides pure LLM interaction without local system access. While it cannot access local files or run local tools, it can still perform complex reasoning and access any cloud-based tools and MCP servers according to the capabilities of the model and the capabilities that may be provided to it by the LLM provider.

Key characteristics:

  • No access to local filesystem (cannot read or write local files)

  • Cannot run local tools or commands

  • Can access cloud-based tools and MCP servers provided by the LLM provider

  • Performs request-response interactions

  • Does not currently maintain conversation state across invocations (not yet implemented)

  • Does not currently support automatic session resumption (not yet implemented)

For tasks requiring local filesystem access or locally-configured tools, use the ‘agent` cog instead.

Defined Under Namespace

Classes: Config, Input, MaxTokensExceededError, Output, Session

Constant Summary collapse

ANTHROPIC_DEFAULT_MAX_TOKENS =

Anthropic always sends a max_tokens value in the request payload with this fallback when the model metadata doesn’t specify one (see ruby_llm’s Anthropic::Chat#build_base_payload).

4096

Instance Attribute Summary collapse

Attributes inherited from Roast::Cog

#name, #output

Instance Method Summary collapse

Methods inherited from Roast::Cog

#anonymous?, config_class, #failed?, generate_fallback_name, #initialize, input_class, #run!, #skipped?, #started?, #stopped?, #succeeded?, #type, #wait

Constructor Details

This class inherits a constructor from Roast::Cog

Instance Attribute Details

#config ⇒ Object (readonly)

The configuration object for this chat cog instance

: Roast::Cogs::Chat::Config



34
35
36
 
# File 'lib/roast/cogs/chat.rb', line 34

def config
  @config
end

Instance Method Details

#execute(input) ⇒ Object

Execute the chat completion with the given input and return the output

: (Input) -> Output



39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/roast/cogs/chat.rb', line 39

def execute(input)
  chat = ruby_llm_context.chat(
    model: config.valid_model,
    provider: config.valid_provider!,
    assume_model_exists: !config.verify_model_exists?,
  )
  input.valid_session&.apply!(chat)
  chat = chat.with_temperature(config.valid_temperature) if config.valid_temperature
  num_existing_messages = chat.messages.length

  response = chat.ask(input.valid_prompt!)
  chat.messages[num_existing_messages..].each do |message|
    case message.role
    when :user
      Event << { block: { header: "USER PROMPT", content: message.content } } if config.show_prompt?
    when :assistant
      Event << { block: { header: "LLM RESPONSE", content: message.content } } if config.show_response?
    else
      # No other message types are expected, but let's show them if they do appear
      # but only the user has requested some form of output
      Event << { block: { header: "UNKNOWN", content: message.content } } if config.show_prompt? || config.show_response?
    end
  end
  if config.show_stats?
    temperature = chat.instance_variable_get(:@temperature)
    lines = ["Model: #{response.model_id}"]
    lines << "Temperature: #{format("%0.2f", temperature)}" if temperature
    lines << "Input Tokens: #{response.input_tokens}"
    lines << "Output Tokens: #{response.output_tokens}"
    Event << { block: { header: "LLM STATS", content: lines.join("\n") } }
  end

  verify_response_not_truncated!(response)

  Output.new(Session.from_chat(chat), response.content)
end