Class: Html2rss::AutoSource::Discovery::DomClustering

Inherits:
Object
  • Object
show all
Defined in:
lib/html2rss/auto_source/discovery/dom_clustering.rb,
lib/html2rss/auto_source/discovery/dom_clustering/group_scorer.rb,
lib/html2rss/auto_source/discovery/dom_clustering/overlap_resolver.rb

Overview

Discovers repeated content card nodes in anchorless or classless DOM trees by evaluating class groups first and falling back to 1-level tag structure signatures.

Defined Under Namespace

Classes: GroupScorer, OverlapResolver

Constant Summary collapse

LAYOUT_TAG_NAMES =

Node tags considered layout containers

Set['div', 'section', 'article', 'li', 'ul', 'ol'].freeze
EXCLUDED_TAGS =

HTML/layout tags excluded from candidate nodes (owned by Html::Navigator).

Html2rss::Html::Navigator::CLUSTER_EXCLUDED_TAGS

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(parsed_body, minimum_frequency:) ⇒ DomClustering

Returns a new instance of DomClustering.

Parameters:

  • parsed_body (Nokogiri::HTML::Document)
  • minimum_frequency (Integer) 


29
30
31
32
33
 
# File 'lib/html2rss/auto_source/discovery/dom_clustering.rb', line 29

def initialize(parsed_body, minimum_frequency:)
  @parsed_body = parsed_body
  @minimum_frequency = minimum_frequency
  @cache = {}.compare_by_identity
end

Class Method Details

.call(parsed_body, minimum_selector_frequency:) ⇒ Array<Nokogiri::XML::Node>

Clusters elements in parsed_body and returns the best set of content card nodes.

Parameters:

  • parsed_body (Nokogiri::HTML::Document) —

    parsed HTML document

  • minimum_selector_frequency (Integer) —

    minimum frequency for candidate groups

Returns:

  • (Array<Nokogiri::XML::Node>) —

    candidate nodes of the top-scoring cluster



22
23
24
 
# File 'lib/html2rss/auto_source/discovery/dom_clustering.rb', line 22

def call(parsed_body, minimum_selector_frequency:)
  new(parsed_body, minimum_frequency: minimum_selector_frequency).call
end

Instance Method Details

#call ⇒ Array<Nokogiri::XML::Node>

Returns:

  • (Array<Nokogiri::XML::Node>) 


36
37
38
39
40
41
 
# File 'lib/html2rss/auto_source/discovery/dom_clustering.rb', line 36

def call
  class_result = cluster_by_class
  return class_result unless class_result.empty?

  cluster_by_structure
end