Module: Automatic::FeedParser

Defined in:
lib/automatic/feed_parser.rb

Class Method Summary collapse

Class Method Details

.get_url(url) ⇒ Object

Fetch a URL and parse it as a feed. Validation is off, because feeds in the wild frequently are not valid and are still readable.

Fetching goes through Automatic::Http, which is where the scheme allowlist, the timeouts and the redirect limit live.



24
25
26
27
28
29
 
# File 'lib/automatic/feed_parser.rb', line 24

def self.get_url(url)
  return if url.nil?

  Automatic::Log.puts('info', "Parsing Feed: #{url}")
  RSS::Parser.parse(Automatic::Http.read(url), false)
end

.parse_html(html) ⇒ Object

Build a feed whose items are the links of an HTML document. This is how a page that publishes no feed enters the pipeline.

nokogiri is required here rather than at the top of the file: it is the only thing in the framework that wants an HTML parser, it is an optional dependency rather than a runtime one, and requiring automatic must neither load one nor need one installed. Only the plugins that call this method -- SubscriptionLink and SubscriptionTumblr -- do. See doc/POLICY.md sections 2.5 and 9.1.



40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/automatic/feed_parser.rb', line 40

def self.parse_html(html)
  Automatic.require_optional(
    'nokogiri',
    needed_by: 'Automatic::FeedParser.parse_html, used by SubscriptionLink ' \
               'and SubscriptionTumblr'
  )

  RSS::Maker.make('2.0') do |maker|
    maker.xml_stylesheets.new_xml_stylesheet
    maker.channel.title = 'Automatic Ruby'
    maker.channel.description = 'Automatic::FeedParser'
    maker.channel.link = 'https://github.com/id774/automaticruby'
    maker.items.do_sort = true

    doc = Nokogiri::HTML(html)
    (doc / :a).each do |link|
      next if link[:href].nil?

      item = maker.items.new_item
      item.title = 'Automatic Ruby'
      item.link = link[:href]
      item.date = Time.now
      item.description = ''
    end
  end
end