Class: Ask::Agent::StreamTransforms::TextBuffer

Inherits:
Base
  • Object
show all
Defined in:
lib/ask/agent/stream_transforms/text_buffer.rb

Overview

Buffers rapid text deltas into larger contiguous chunks.

LLM streaming often produces many tiny deltas (1–5 characters each), especially over high-latency connections. This transform coalesces them into chunks of at least min_size characters, reducing the number of UI updates, log entries, or event emissions.

Only text content is buffered. Non-content chunks (tool calls, finish signals, usage data) pass through immediately.

Examples:

Buffer until at least 100 characters

pipeline.use :text_buffer, min_size: 100

Instance Method Summary collapse

Constructor Details

#initialize(min_size: 50) ⇒ TextBuffer

Returns a new instance of TextBuffer.



19
20
21
22
23
24
 
# File 'lib/ask/agent/stream_transforms/text_buffer.rb', line 19

def initialize(min_size: 50)
  @min_size = min_size
  @buffer = +""
  @pending_usage = nil
  @pending_finish = nil
end

Instance Method Details

#call(chunk, &block) ⇒ Object 



26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
 
# File 'lib/ask/agent/stream_transforms/text_buffer.rb', line 26

def call(chunk, &block)
  if chunk.content.to_s.strip.length > 0
    @buffer << chunk.content

    if @buffer.length >= @min_size
      emit_buffer(&block)
    end
  else
    # Flush buffer before non-content chunks
    emit_buffer(&block)

    # Track metadata to emit after flush
    if chunk.finish_reason
      @pending_finish = chunk.finish_reason
    end
    if chunk.usage
      @pending_usage = chunk.usage
    end

    # Pass through non-content chunks (tool calls, etc.)
    yield chunk
  end
end

#finish(&block) ⇒ Object 



50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
# File 'lib/ask/agent/stream_transforms/text_buffer.rb', line 50

def finish(&block)
  emit_buffer(&block) if @buffer.length > 0

  # Emit pending metadata if any
  if @pending_finish || @pending_usage
    yield Ask::Chunk.new(
      content: nil,
      tool_calls: nil,
      finish_reason: @pending_finish,
      usage: @pending_usage,
      raw: nil,
      thinking: nil
    )
    @pending_finish = nil
    @pending_usage = nil
  end
end