Module: Relaton::Ietf::Scraper

Extended by:
Scraper
Included in:
Scraper
Defined in:
lib/relaton/ietf/scraper.rb

Overview

Scraper module

Constant Summary collapse

IDS = 
"https://raw.githubusercontent.com/relaton/relaton-data-ids/refs/heads/data-v2/"
RFC = 
"https://raw.githubusercontent.com/relaton/relaton-data-rfcs/refs/heads/data-v2/"
RSS = 
"https://raw.githubusercontent.com/relaton/relaton-data-rfcsubseries/refs/heads/data-v2/"

Instance Method Summary collapse

Instance Method Details

#scrape_page(text) ⇒ RelatonIetf::IetfBibliographicItem

Parameters:

  • text (String)

Returns:

  • (RelatonIetf::IetfBibliographicItem) 


15
16
17
18
19
20
21
22
23
24
 
# File 'lib/relaton/ietf/scraper.rb', line 15

def scrape_page(text)
  # Remove initial "IETF " string if specified
  ref = text.gsub(/^IETF /, "")
  # ref.sub!(/(?<=^(?:RFC|BCP|FYI|STD))\s(\d+)/) { $1.rjust 4, "0" }
  rfc_item ref
rescue Timeout::Error, Errno::EINVAL, Errno::ECONNRESET, EOFError,
      Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError,
      Net::ProtocolError, SocketError
  raise Relaton::RequestError, "No document found for #{ref} reference"
end