Class: Unmagic::Browser::Driver::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/unmagic/browser/driver/base.rb

Overview

What every driver has to be able to do, and the parts none of them need to write twice.

A driver answers two questions: how do I get a browser, and what's the cheapest way to grab a single page? Subclasses answer the first with #connect_browser and #disconnect_browser. The second is answered here, for free, by opening a throwaway page on a shared connection — Cloudflare overrides it because it has a REST API that needs no browser at all.

Direct Known Subclasses

Cloudflare, Local, Remote

Constant Summary collapse

DEFAULT_TIMEOUT =

Seconds a one-off page grab waits for the page to settle.

30
WAIT_UNTIL =

When a page counts as loaded. networkidle2 waits out the XHR that a JS-rendered page fires after load, which is the whole reason to run a real browser instead of fetching HTML.

"networkidle2"

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Base

Returns a new instance of Base.



24
25
26
27
 
# File 'lib/unmagic/browser/driver/base.rb', line 24

def initialize
  @scratch_mutex = Mutex.new
  @scratch = nil
end

Instance Method Details

#close ⇒ void

This method returns an undefined value.

Drop the shared connection one-off grabs run on. Sessions are unaffected — each holds its own browser.



76
77
78
79
80
81
82
83
 
# File 'lib/unmagic/browser/driver/base.rb', line 76

def close
  scratch = nil
  @scratch_mutex.synchronize do
    scratch = @scratch
    @scratch = nil
  end
  scratch&.close
end

#html(url) ⇒ String

Returns the page's HTML, after JavaScript has run.

Parameters:

  • url (String) —

    the page to render

Returns:

  • (String) —

    the page's HTML, after JavaScript has run



44
45
46
 
# File 'lib/unmagic/browser/driver/base.rb', line 44

def html(url)
  on_scratch_page(url) { |page| page.content }
end

#markdown(url) ⇒ String

Returns the page as Markdown.

Parameters:

  • url (String) —

    the page to render

Returns:

  • (String) —

    the page as Markdown



38
39
40
 
# File 'lib/unmagic/browser/driver/base.rb', line 38

def markdown(url)
  Unmagic::Browser::Markdown.from_html(html(url))
end

#name ⇒ String

Returns a short description used in log lines.

Returns:

  • (String) —

    a short description used in log lines



86
87
88
 
# File 'lib/unmagic/browser/driver/base.rb', line 86

def name
  self.class.name.split("::").last.downcase
end

#open_connection ⇒ Connection

Open a browser of this driver's kind, for a caller who'll hold it.

Returns:



32
33
34
 
# File 'lib/unmagic/browser/driver/base.rb', line 32

def open_connection
  Failures.wrap("connecting to the browser") { Connection.new(self, connect_browser) }
end

#pdf(url) ⇒ String

Returns PDF bytes.

Parameters:

  • url (String) —

    the page to render

Returns:

  • (String) —

    PDF bytes



68
69
70
 
# File 'lib/unmagic/browser/driver/base.rb', line 68

def pdf(url)
  on_scratch_page(url) { |page| page.pdf }
end

#screenshot(url, full_page: false, selector: nil) ⇒ String

Returns PNG bytes.

Parameters:

  • url (String) —

    the page to render

  • full_page (Boolean) (defaults to: false) —

    capture the whole scrollable page

  • selector (String, nil) (defaults to: nil) —

    capture just the first element matching this CSS

Returns:

  • (String) —

    PNG bytes

Raises:



53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/unmagic/browser/driver/base.rb', line 53

def screenshot(url, full_page: false, selector: nil)
  on_scratch_page(url) do |page|
    if selector
      element = page.query_selector(selector)
      raise Error::NotFound, "No element matches #{selector.inspect} on #{url}." unless element

      element.screenshot(type: "png")
    else
      page.screenshot(type: "png", full_page: full_page)
    end
  end
end