Class: Html2rss::AutoSource::Scraper::NativeFeed

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/html2rss/auto_source/scraper/native_feed.rb

Overview

Promotes a page's native RSS/Atom feed via Syndication::Discovery + Syndication::Parser.

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(parsed_body, url:, request_session: nil, **_opts) ⇒ NativeFeed

Returns a new instance of NativeFeed.

Parameters:

Options Hash (**_opts):

  • :_reserved (Object) —

    reserved for future scraper-specific options



45
46
47
48
49
 
# File 'lib/html2rss/auto_source/scraper/native_feed.rb', line 45

def initialize(parsed_body, url:, request_session: nil, **_opts)
  @parsed_body = parsed_body
  @url = Html2rss::Url.from_absolute(url)
  @request_session = request_session
end

Class Method Details

.articles?(parsed_body) ⇒ Boolean

Shallow claim: head advertises a syndication alternate, or body looks feed-capable enough that path discovery is worth a follow-up slot.

Parameters:

  • parsed_body (Nokogiri::HTML::Document, nil)

Returns:

  • (Boolean) 


27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/html2rss/auto_source/scraper/native_feed.rb', line 27

def self.articles?(parsed_body)
  return false unless parsed_body.is_a?(Nokogiri::HTML::Document)

  return true if ::Html2rss::Html::FeedLink.from_document(parsed_body).any?

  parsed_body.css('head link[rel~="alternate"][href]').any? do |node|
    href = node['href'].to_s
    type = node['type'].to_s.downcase
    type.include?('rss') || type.include?('atom') || href.match?(/rss|atom|feed|\.xml/i)
  end
end

.options_key ⇒ Symbol

Returns scraper config key.

Returns:

  • (Symbol) —

    scraper config key



12
 
# File 'lib/html2rss/auto_source/scraper/native_feed.rb', line 12

def self.options_key = :native_feed

.request_slots(_opts = {}) ⇒ Integer

Returns follow-up request slots (discovery probes + feed fetch share the budget).

Parameters:

  • _opts (Hash) (defaults to: {}) —

    unused options

Returns:

  • (Integer) —

    follow-up request slots (discovery probes + feed fetch share the budget)



17
18
19
 
# File 'lib/html2rss/auto_source/scraper/native_feed.rb', line 17

def self.request_slots(_opts = {})
  1
end

Instance Method Details

#each {|article| ... } ⇒ Enumerator, void

Yield Parameters:

  • article (Hash{Symbol => Object})

Returns:

  • (Enumerator, void) 


54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/html2rss/auto_source/scraper/native_feed.rb', line 54

def each(&)
  return enum_for(:each) unless block_given?

  articles = fetch_articles
  if articles.empty?
    Log.info("#{self.class}: host=#{url.host} item_count=0 fallback=true")
    return
  end

  Log.info("#{self.class}: host=#{url.host} item_count=#{articles.size} fallback=false")
  articles.each(&)
end