Module: Unmagic::Browser::Markdown

Defined in:
lib/unmagic/browser/markdown.rb

Overview

Turns a rendered page into Markdown.

This is the fallback for drivers with no Markdown endpoint of their own — Cloudflare renders its own server-side, and its output won't match this byte for byte. What both guarantee is the same shape: the page's content without the furniture.

Constant Summary collapse

NOISE =

Elements that carry no content a reader wants, and would otherwise show up as noise or as raw JavaScript.

"script, style, noscript, template, svg, iframe, link, meta, head, " \
"nav, header, footer, [aria-hidden=true], [hidden]"
CONTENT_ROOTS =

Where a page's own content usually lives, most specific first.

["main", "article", "[role=main]", "body"].freeze

Class Method Summary collapse

Class Method Details

.from_html(html) ⇒ String

Convert rendered HTML to Markdown.

Parameters:

  • html (String) —

    the page's HTML, after JavaScript has run

Returns:

  • (String) 


28
29
30
31
32
 
# File 'lib/unmagic/browser/markdown.rb', line 28

def from_html(html)
  document = Nokogiri::HTML(html.to_s)
  document.css(NOISE).remove
  convert(content_root(document))
end