Class: Rubino::UI::StreamingMarkdown

Inherits:
Object
  • Object
show all
Defined in:
lib/rubino/ui/streaming_markdown.rb

Overview

Incremental block splitter for streamed markdown.

The model streams an assistant message token-by-token; we want to render COMPLETED markdown blocks above the composer as soon as they finish, while showing the still-incoming (incomplete) block raw in the live region. This buffer accumulates streamed text and decides where one block ends and the next begins, so CLI can render+commit a finished block and leave only the in-progress tail live.

Block boundary detection — a small line-oriented fence state machine (the mainstream approach used by md2term / mdterm / Glamour-style streamers: you must NOT render a fenced code block until its closing “‘ arrives, or half-open fences render as garbage):

* Lines are split on "\n". A line is "complete" once its terminating "\n"
  has been seen; the trailing remainder (no "\n" yet) is the live tail.
* A line matching ^\s*``` toggles the fence state.
    - Entering a fence STARTS a code block (the fence line joins it).
    - Leaving a fence ENDS the code block (the closing fence joins it);
      the block is reported complete.
* While INSIDE a fence, blank lines do NOT split — code keeps its blanks.
* While OUTSIDE a fence, a blank line ENDS the current prose block. The
  blank line itself is consumed as the separator (not re-emitted).

API:

feed(text)  -> Array<String>  newly-completed block texts (state advances)
tail        -> String         the current incomplete block, raw (live)
flush       -> String|nil     the remaining buffered block on stream end
                              (an unclosed fence is returned so the caller
                              can emit it as plain text — never lost)

Constant Summary collapse

FENCE_RE = 
/\A\s*```/
FENCE_OPEN_RE =

An OPENING fence captures its run of backticks (≥3) and any info string (the language tag) that follows. The CommonMark rule we lean on: a fenced block is closed only by a bare fence — no info string — of AT LEAST as many backticks. That keeps a nested “‘ruby inside an outer “`markdown from being mistaken for the close (it carries an info string), so the whole wrapped block stays one unit instead of mis-toggling (#264).

/\A\s*(`{3,})\s*(\S.*)?\z/
FENCE_CLOSE_RE = 
/\A\s*(`{3,})\s*\z/
MARKDOWN_FENCE_LANGS =

Info strings that mean “this fence WRAPS markdown” — the model boxed a whole answer (which itself contains nested “‘lang fences) in an outer “`markdown / “`md. For those, and ONLY those, we track fence-nesting DEPTH: a nested fence’s bare close must not be mistaken for the wrapper’s close (T1), so the whole wrapped answer stays ONE block. The renderer re-renders such a body AS markdown (MarkdownRenderer::MARKDOWN_FENCE_LANGS).

%w[markdown md].freeze
LIST_ITEM_RE =

An ordered (“1. ”, “2) ”) or unordered (“- ”, “* ”, “+ ”) list item. Used so a loose list (blank lines BETWEEN items) is kept as ONE block instead of being split per-item: each split item was re-rendered on its own, and kramdown restarts ordered numbering at 1 for every block, which produced the “1. Mercury / 1. Venus / 1. Earth” off-by-one (B4).

/\A\s*(?:[-*+]|\d+[.)])\s/
TABLE_SEP_RE =

A GFM pipe-table separator row, e.g. “|—|:–:|—|” or “—|—”. The EXACT regex MarkdownRenderer uses to detect a table (markdown_renderer.rb) — DRY: a separator row is the single unambiguous “this is a table” signal, so the splitter and the renderer must agree on what one looks like.

MarkdownRenderer::TABLE_SEP_RE

Instance Method Summary collapse

Constructor Details

#initialize ⇒ StreamingMarkdown

Returns a new instance of StreamingMarkdown.



65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/rubino/ui/streaming_markdown.rb', line 65

def initialize
  @pending = +""   # un-newlined remainder (the live tail-in-progress line)
  @block   = []    # completed lines accumulated for the current block
  @in_fence = false
  @fence_len = 0 # backtick count of the OPEN fence (close needs ≥ this many)
  # When the OPEN fence is a ```markdown / ```md wrapper, we track nesting
  # DEPTH (starts at 1 on open): nested opening fences ++ it, bare closes of
  # ≥ @fence_len -- it, and the block completes only when it returns to 0.
  # nil = the open fence is a PLAIN code fence (no nesting; closes on the
  # first bare fence of ≥ @fence_len), the CommonMark default (T1).
  @fence_depth = nil
  @in_list  = false # current block is a markdown list (keep loose items together)
  @blanks   = 0     # blank lines buffered inside a list, re-emitted iff it continues
  # A GFM pipe table is its OWN block type (like a fence): once the
  # separator row is consumed after a header-ish row we hold every
  # following pipe row in the block and NEVER show raw pipes live — the
  # CLI re-renders the completed-rows-so-far as a fitted partial table
  # (Option B) instead. A blank line or a non-pipe line ENDS the table.
  @in_table = false
end

Instance Method Details

#feed(text) ⇒ Object

Accumulate streamed text; return the list of block texts that became COMPLETE as a result of this feed (possibly empty). Advances state.



106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/rubino/ui/streaming_markdown.rb', line 106

def feed(text)
  return [] if text.nil? || text.empty?

  @pending << text
  completed = []

  while (idx = @pending.index("\n"))
    line = @pending[0...idx]
    @pending = @pending[(idx + 1)..] || +""
    block = consume_line(line)
    completed << block if block
  end

  completed
end

#flush ⇒ Object

Drain the remainder on stream end. Promotes any un-newlined remainder to a final line, then returns the buffered block text (or nil if empty). An unclosed fence is returned all the same — the caller emits it as plain so output is never dropped.



160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
 
# File 'lib/rubino/ui/streaming_markdown.rb', line 160

def flush
  unless @pending.empty?
    @block << @pending
    @pending = +""
  end
  return nil if @block.empty?

  text = @block.join("\n")
  @block = []
  @in_fence = false
  @fence_len = 0
  @fence_depth = nil
  @in_list = false
  @blanks = 0
  @in_table = false
  text
end

#in_table? ⇒ Boolean

True while the in-flight block is a GFM pipe table (header + separator consumed, rows still arriving). The CLI uses this to paint a fitted partial table in the live region instead of leaking raw ‘| … |` rows.

Returns:

  • (Boolean) 


89
90
91
 
# File 'lib/rubino/ui/streaming_markdown.rb', line 89

def in_table?
  @in_table
end

#live_tail(rows = 1) ⇒ Object

The in-progress tail to show live (raw): the LAST rows lines of the in-flight block — its most recent already-newlined lines plus the un-newlined remainder. Newline-joined; the live region renders one row per line.

Why a rolling window and not the whole #tail: the live region must stay bounded (a long open fence/table must never push the prompt off-screen), so we keep “only the last block can change” (Textual/Rich, Streamdown, Glamour-style streamers) but show a FEW trailing lines instead of just the one being typed — a long list block used to vanish line-by-line as each item completed, leaving a single flickering raw line until the whole block committed (#127). Earlier lines stay buffered and the block still snaps to rendered markdown the moment it completes.



144
145
146
147
148
149
150
151
152
153
154
 
# File 'lib/rubino/ui/streaming_markdown.rb', line 144

def live_tail(rows = 1)
  # While a table is in flight the raw `| col | col |` rows must NEVER be
  # shown live (they wrap mid-cell, no borders): the CLI paints a fitted
  # partial table (from #table_rows_so_far) instead. Return empty here so
  # no caller can leak raw pipes regardless of which branch it took.
  return "" if @in_table

  lines = @block.last(rows)
  lines += [@pending] unless @pending.empty?
  lines.last(rows).join("\n")
end

#table_rows_so_far ⇒ Object

The table-so-far as COMPLETED lines (header, separator, and every fully-arrived data row) — the in-flight last partial row (the un-newlined shows a half-typed row. Empty unless a table block is in flight. Feeding MarkdownRenderer this subset yields a correctly-bordered growing table.



98
99
100
101
102
 
# File 'lib/rubino/ui/streaming_markdown.rb', line 98

def table_rows_so_far
  return [] unless @in_table

  @block.dup
end

#tail ⇒ Object

The current incomplete block as raw text: any lines already buffered for the in-progress block plus the un-newlined remainder. Shown live; it gets re-rendered + committed once its block completes.



125
126
127
128
129
 
# File 'lib/rubino/ui/streaming_markdown.rb', line 125

def tail
  parts = @block.dup
  parts << @pending unless @pending.empty?
  parts.join("\n")
end