Class: Leptris::XML::SAX::Recorder

Inherits:
Object
  • Object
show all
Defined in:
lib/leptris/xml/sax/recorder.rb

Overview

Chunked event recorder (libleptris 1.9.4, issue #585): a SAX transport that buffers events C-side — fixed LeptrisSaxEventRecord entries plus a packed string arena — and drains them in one bulk read per fed chunk. The handler receives the identical calls the callback API makes; only the transport changes, and the callback count becomes O(chunks), not O(events): through FFI, per-event C-to-Ruby dispatch cost more than the parse itself.

Leptris::XML::SAX::Recorder.parse(xml) do |kind, name, text, attrs|
  # kind: :start_element, :characters, ... (see KINDS)
end

Constant Summary collapse

KINDS = 
{
  Leptris::XML::FFI::SAX_EVENT_START_DOCUMENT => :start_document,
  Leptris::XML::FFI::SAX_EVENT_END_DOCUMENT => :end_document,
  Leptris::XML::FFI::SAX_EVENT_START_ELEMENT => :start_element,
  Leptris::XML::FFI::SAX_EVENT_END_ELEMENT => :end_element,
  Leptris::XML::FFI::SAX_EVENT_CHARACTERS => :characters,
  Leptris::XML::FFI::SAX_EVENT_COMMENT => :comment,
  Leptris::XML::FFI::SAX_EVENT_CDATA => :cdata,
  Leptris::XML::FFI::SAX_EVENT_PI => :pi,
  Leptris::XML::FFI::SAX_EVENT_START_PREFIX => :start_prefix,
  Leptris::XML::FFI::SAX_EVENT_END_PREFIX => :end_prefix,
  Leptris::XML::FFI::SAX_EVENT_ERROR => :error,
}.freeze
KIND_BY_CODE =

Kind symbol per code byte — Array indexing in the drain loop, not a Hash lookup per event.

Array.new(KINDS.size) { |code| KINDS[code] }.freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(handle) ⇒ Recorder

Returns a new instance of Recorder.



62
63
64
 
# File 'lib/leptris/xml/sax/recorder.rb', line 62

def initialize(handle)
  @handle = handle
end

Class Method Details

.open ⇒ Object 



36
37
38
39
40
41
42
 
# File 'lib/leptris/xml/sax/recorder.rb', line 36

def self.open
  raw = Leptris::XML::FFI.leptris_sax_recorder_new
  if raw.null?
    raise Leptris::XML::Error, "leptris_sax_recorder_new failed"
  end
  new(raw)
end

.parse(xml_or_io, kinds: nil) ⇒ Object

One-shot parse over a complete document, yielding Record structs. kinds filters the drain (see #each_event) — pass e.g. kinds: [:start_element] to slice strings only for the events you consume.



48
49
50
51
52
53
54
55
56
57
58
59
60
 
# File 'lib/leptris/xml/sax/recorder.rb', line 48

def self.parse(xml_or_io, kinds: nil)
  recorder = open
  begin
    if xml_or_io.respond_to?(:read)
      recorder.feed_stream(xml_or_io, kinds: kinds) { |*args| yield(*args) }
    else
      recorder.feed(xml_or_io.to_s, final: true)
      recorder.each_event(*Array(kinds)) { |*args| yield(*args) }
    end
  ensure
    recorder.free
  end
end

Instance Method Details

#each_event(*kinds) ⇒ Object

Drains the current chunk's records: bulk-reads the record array and the packed arena (two FFI calls), then slices every string from the arena in Ruby — no per-string FFI. With kinds, records of other kinds are skipped BEFORE any string is sliced: an unwanted event costs one Array read, and cost scales with what the consumer asked for, not with the document's event mix.



97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
 
# File 'lib/leptris/xml/sax/recorder.rb', line 97

def each_event(*kinds)
  wanted = kinds.empty? ? nil : wanted_set(kinds)
  count_ptr = ::FFI::MemoryPointer.new(:size_t)
  len_ptr = ::FFI::MemoryPointer.new(:size_t)
  begin
    records_ptr = Leptris::XML::FFI.leptris_sax_recorder_records(
      @handle, count_ptr)
    return self if records_ptr.null? || count_ptr.read_uint64.zero?

    arena_ptr = Leptris::XML::FFI.leptris_sax_recorder_arena(
      @handle, len_ptr)
    arena = arena_ptr.read_bytes(len_ptr.read_uint64)
    count = count_ptr.read_uint64

    # The whole record block in ONE read: read_bytes + String#unpack
    # (C-fast) — two FFI calls per chunk total (records + arena),
    # the true bulk discipline. Template per record: kind byte,
    # 7 pad, then the eight uint32 fields (little-endian hosts —
    # the layout the header freezes for FFI mirrors).
    stride = Leptris::XML::FFI::SaxEventRecord.size
    fields = records_ptr.read_bytes(count * stride)
      .unpack("C x7 V8" * count)

    i = 0
    while i < count
      kind = KIND_BY_CODE[fields[i * 9]]
      if kind && (wanted.nil? || wanted[kind])
        base = i * 9 + 1
        yield kind,
              slice(arena, fields[base], fields[base + 1]),
              slice(arena, fields[base + 2], fields[base + 3]),
              attrs_from(arena, fields[base + 4], fields[base + 5]),
              fields[base + 6], fields[base + 7]
      end
      i += 1
    end
  ensure
    count_ptr.free
    len_ptr.free
  end
  self
end

#feed(chunk, final: false) ⇒ Object

Feed one chunk; records/arena reset at feed entry, so drain after every feed. Returns self.



74
75
76
77
78
79
 
# File 'lib/leptris/xml/sax/recorder.rb', line 74

def feed(chunk, final: false)
  chunk = chunk.dup.force_encoding(Encoding::UTF_8)
  rc = Leptris::XML::FFI.leptris_sax_recorder_feed(
    @handle, chunk, chunk.bytesize, final ? 1 : 0)
  rc
end

#feed_stream(io, chunk_size = 65_536, kinds: nil, &block) ⇒ Object

Feed an IO in chunks, draining events after each chunk.



82
83
84
85
86
87
88
89
 
# File 'lib/leptris/xml/sax/recorder.rb', line 82

def feed_stream(io, chunk_size = 65_536, kinds: nil, &block)
  while (chunk = io.read(chunk_size))
    feed(chunk)
    each_event(*Array(kinds), &block)
  end
  feed("", final: true)
  each_event(*Array(kinds), &block)
end

#free ⇒ Object 



66
67
68
69
70
 
# File 'lib/leptris/xml/sax/recorder.rb', line 66

def free
  return if @handle.nil?
  Leptris::XML::FFI.leptris_sax_recorder_free(@handle)
  @handle = nil
end