Module: Html2rss::Syndication::Discovery

Defined in:
lib/html2rss/syndication/discovery.rb

Overview

Finds a same-origin RSS/Atom URL for a page via head alternates and path guesses.

Ports configs probe_rss path/alternate logic onto RequestSession#follow_up.

Constant Summary collapse

DEFAULT_PATHS =

Common feed path suffixes probed after head rel=alternate hints.

CandidateCatalog::FEED_PATHS

Class Method Summary collapse

Class Method Details

.best_feed_response(page_url:, request_session:, parsed_body: nil, html: nil, extra_paths: [], max_probes: nil) ⇒ Html2rss::RequestService::Response?

Like #best_feed_url but returns the validated syndication response for parsing.

rubocop:disable Metrics/ParameterLists, Metrics/MethodLength -- discovery kwargs stay co-located

Parameters:

  • page_url (String, Html2rss::Url)
  • request_session (Html2rss::RequestSession)
  • parsed_body (Nokogiri::HTML::Document, nil) (defaults to: nil)
  • html (String, nil) (defaults to: nil)
  • extra_paths (Array<String>) (defaults to: [])
  • max_probes (Integer, nil) (defaults to: nil) —

    max candidate GETs (nil = uncapped)

Returns:



51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
# File 'lib/html2rss/syndication/discovery.rb', line 51

def best_feed_response(page_url:, request_session:, parsed_body: nil, html: nil, extra_paths: [],
                       max_probes: nil)
  page = Html2rss::Url.from_absolute(page_url)
  candidates = candidate_urls(page_url: page, parsed_body:, html:, extra_paths:)
  Log.debug(
    "Syndication::Discovery: host=#{page.host} candidate_count=#{candidates.size}"
  )

  first_feedish_response(
    candidates, request_session:, origin_url: page, max_probes:
  ).tap do |response|
    next unless response

    Log.info("Syndication::Discovery: host=#{page.host} selected_feed_url=#{response.url}")
  end
end

.best_feed_url(page_url:, request_session:, parsed_body: nil, html: nil, extra_paths: [], max_probes: nil) ⇒ Html2rss::Url?

Probes candidates lazily and returns the first feedish URL.

rubocop:disable Metrics/ParameterLists -- discovery kwargs stay co-located

Parameters:

  • page_url (String, Html2rss::Url) —

    page or origin URL

  • request_session (Html2rss::RequestSession) —

    shared session for follow-ups

  • parsed_body (Nokogiri::HTML::Document, nil) (defaults to: nil) —

    parsed HTML when available

  • html (String, nil) (defaults to: nil) —

    raw HTML when a document is unavailable

  • extra_paths (Array<String>) (defaults to: []) —

    additional path guesses

  • max_probes (Integer, nil) (defaults to: nil) —

    max candidate GETs (nil = uncapped)

Returns:



32
33
34
35
36
37
 
# File 'lib/html2rss/syndication/discovery.rb', line 32

def best_feed_url(page_url:, request_session:, parsed_body: nil, html: nil, extra_paths: [],
                  max_probes: nil)
  best_feed_response(
    page_url:, request_session:, parsed_body:, html:, extra_paths:, max_probes:
  )&.url
end

.candidate_urls(page_url:, parsed_body: nil, html: nil, extra_paths: []) ⇒ Array<Html2rss::Url>

Ordered candidate feed URLs (head alternates first, then path guesses).

Parameters:

  • page_url (String, Html2rss::Url)
  • parsed_body (Nokogiri::HTML::Document, nil) (defaults to: nil)
  • html (String, nil) (defaults to: nil)
  • extra_paths (Array<String>) (defaults to: [])

Returns:



77
78
79
80
81
 
# File 'lib/html2rss/syndication/discovery.rb', line 77

def candidate_urls(page_url:, parsed_body: nil, html: nil, extra_paths: [])
  page = Html2rss::Url.from_absolute(page_url)
  (alternate_feed_urls(page, parsed_body:, html:) +
    path_guess_urls(page, extra_paths:)).uniq
end

.feedish?(response) ⇒ Boolean

Parameters:

Returns:

  • (Boolean) 


86
87
88
89
90
91
 
# File 'lib/html2rss/syndication/discovery.rb', line 86

def feedish?(response)
  return false unless response
  return false unless successful_status?(response)

  response.feed_response?
end

.page_dir_paths(page_url) ⇒ Array<String>

Directory-relative feed path guesses for a page URL.

Parameters:

Returns:

  • (Array<String>) —

    relative paths



105
106
107
108
109
110
111
 
# File 'lib/html2rss/syndication/discovery.rb', line 105

def page_dir_paths(page_url)
  path = Html2rss::Url.from_absolute(page_url).path
  return [] if path.nil? || path.empty? || path == '/'

  dir = path.sub(%r{/[^/]*$}, '/')
  ["#{dir}feed", "#{dir}rss.xml", "#{dir}atom.xml"]
end