Class: Html2rss::SST::Normalizer

Inherits:
Object
  • Object
show all
Defined in:
lib/html2rss/sst/normalizer.rb

Overview

Sole Nokogiri consumer on the heuristic auto-source path. Builds an immutable SST::Document with parent/depth/chrome indices.

Constant Summary collapse

MAX_NODES =

Hard ceiling for SST node allocations; beyond this we degrade.

5_000
STRIPPED_TAGS =

Tags stripped entirely from the SST.

Set['script', 'style', 'noscript', 'iframe', 'svg', 'template'].freeze
SEMANTIC_DEGRADE_TAGS =

Kept when MAX_NODES forces semantic-tag-only degrade.

Set['html', 'body', 'article', 'section', 'li', 'tr', 'div', 'a',
'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'time', 'img', 'p',
'ul', 'ol', 'main'].freeze
RAW_ATTR_KEEP =

Attribute names preserved in Attrs#raw for extraction.

/
  \A(
    data- |
    aria- |
    class |
    id |
    href |
    src |
    srcset |
    style |
    datetime |
    itemprop |
    type |
    category |
    categories |
    tag |
    tags |
    topic |
    topics |
    section |
    sections |
    label |
    labels |
    theme |
    themes |
    subject |
    subjects
  )
/xi

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(parsed_body) ⇒ Normalizer

Returns a new instance of Normalizer.

Parameters:

  • parsed_body (Nokogiri::HTML::Document, Nokogiri::XML::Node) 


80
81
82
83
84
85
86
87
 
# File 'lib/html2rss/sst/normalizer.rb', line 80

def initialize(parsed_body)
  @parsed_body = parsed_body
  @node_count = 0
  @degraded = false
  @parents = {}.compare_by_identity
  @depths = {}.compare_by_identity
  @ignored_chrome = {}.compare_by_identity
end

Class Method Details

.call(input) ⇒ Document

Parameters:

  • input (String, Nokogiri::HTML::Document, Nokogiri::XML::Node) —

    HTML string or already-parsed node (String is parsed once here)

Returns:

Raises:

  • (ArgumentError) —

    when input is nil or unsupported



59
60
61
62
63
 
# File 'lib/html2rss/sst/normalizer.rb', line 59

def call(input)
  raise ArgumentError, 'input is required' if input.nil?

  new(coerce(input)).call
end

Instance Method Details

#call ⇒ Document

Returns:

Raises:

  • (ArgumentError) 


91
92
93
94
95
96
97
98
 
# File 'lib/html2rss/sst/normalizer.rb', line 91

def call
  root_nk = @parsed_body.respond_to?(:root) ? (@parsed_body.at_css('html') || @parsed_body.root) : @parsed_body
  root = normalize_element(root_nk, parent: nil, depth: 0, path: '', chrome: false)
  raise ArgumentError, 'SST Normalizer produced an empty tree' unless root

  index = Index.new(root:, parents: @parents, depths: @depths, ignored_chrome: @ignored_chrome)
  Document.build(root:, index:, degraded: @degraded, node_count: @node_count)
end