Class: LlmScraper::ContentFetchers::Local

Inherits:

Base

Object
Base
LlmScraper::ContentFetchers::Local

show all

Defined in:: lib/llm_scraper/content_fetchers/local.rb

Instance Method Summary collapse

#fetch(url, spa: false) ⇒ String

Cleaned plain text.
#initialize(config = LlmScraper.configuration) ⇒ Local constructor

A new instance of Local.

Constructor Details

#initialize(config = LlmScraper.configuration) ⇒ `Local`

Returns a new instance of Local.



6
7
8

# File 'lib/llm_scraper/content_fetchers/local.rb', line 6

def initialize(config = LlmScraper.configuration)
  @config = config
end

Instance Method Details

#fetch(url, spa: false) ⇒ `String`

Returns cleaned plain text.

Parameters:

url (String)
spa (Boolean) (defaults to: false) —

use Ferrum for JS-rendered pages

Returns:

(String) —

cleaned plain text



13
14
15

# File 'lib/llm_scraper/content_fetchers/local.rb', line 13

def fetch(url, spa: false)
  spa ? fetch_spa(url) : fetch_static(url)
end