Class: Html2rss::Html::ArticleExtractor::CategoryExtractor

Inherits:
Object
  • Object
show all
Defined in:
lib/html2rss/html/article_extractor/category_extractor.rb

Overview

CategoryExtractor is responsible for extracting categories from HTML elements by looking for CSS class names containing common category-related terms.

Constant Summary collapse

CATEGORY_TERMS =

Shared category vocabulary (owned by Html2rss::Html::ArticleRules::Category).

ArticleRules::Category::CATEGORY_TERMS
CATEGORY_ATTR_PATTERN =

Shared category attribute/class pattern (owned by Html2rss::Html::ArticleRules::Category).

ArticleRules::Category::CATEGORY_ATTR_PATTERN
CATEGORY_SELECTORS =

CSS selectors to find elements with category-related class names or data attributes

CATEGORY_TERMS.flat_map do |term|
  ["[class*=\"#{term}\"]", "[data-#{term}]", "[#{term}]"]
end.freeze

Class Method Summary collapse

Class Method Details

.call(article_tag) ⇒ Array<String>

Extracts categories from the given article tag by looking for elements with class names containing common category-related terms.

Parameters:

  • article_tag (Nokogiri::XML::Element) —

    The article element to extract categories from

Returns:

  • (Array<String>) —

    Array of category strings, empty if none found



26
27
28
29
30
31
32
 
# File 'lib/html2rss/html/article_extractor/category_extractor.rb', line 26

def self.call(article_tag)
  return [] unless article_tag

  extract_all_categories(article_tag)
    .map(&:strip)
    .reject(&:empty?)
end

.extract_all_categories(article_tag) ⇒ Set<String>

Optimized single DOM traversal that extracts all category types.

Parameters:

  • article_tag (Nokogiri::XML::Element) —

    The article element

Returns:

  • (Set<String>) —

    Set of category strings



39
40
41
42
43
44
45
46
 
# File 'lib/html2rss/html/article_extractor/category_extractor.rb', line 39

def self.extract_all_categories(article_tag)
  Set.new.tap do |categories|
    article_tag.css(CATEGORY_SELECTORS.join(',')).each do |element|
      extract_text_categories!(categories, element) if ArticleRules::Category.class_match?(element['class'])
      extract_element_data_categories!(categories, element)
    end
  end
end

.extract_element_data_categories!(categories, element) ⇒ void

This method returns an undefined value.

Extracts categories from data attributes of a single element.

Parameters:

  • categories (Set<String>) —

    Accumulator set

  • element (Nokogiri::XML::Element) —

    metadata element that may contain category links



54
55
56
57
58
59
60
 
# File 'lib/html2rss/html/article_extractor/category_extractor.rb', line 54

def self.extract_element_data_categories!(categories, element)
  element.attributes.each_value do |attr|
    next unless ArticleRules::Category.attr_name_match?(attr.name)

    ArticleRules::Category.add_text!(categories, attr.value)
  end
end

.extract_text_categories!(categories, element) ⇒ void

This method returns an undefined value.

Extracts text-based categories from elements, splitting content into discrete values.

Parameters:

  • categories (Set<String>) —

    Accumulator set

  • element (Nokogiri::XML::Element) —

    metadata element whose text may contain delimiters



68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/html2rss/html/article_extractor/category_extractor.rb', line 68

def self.extract_text_categories!(categories, element)
  if element.name == 'a'
    add_text_to_categories!(categories, element)
    return
  end

  anchors = element.css('a')

  if anchors.any?
    anchors.each { |node| add_text_to_categories!(categories, node) }
  else
    ArticleRules::Category.add_split_text!(categories, element.text)
  end
end