Class: ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::HTML

Inherits:
Internal::Type::BaseModel show all
Defined in:
lib/context_dev/models/batch_submit_params.rb,
sig/context_dev/models/batch_submit_params.rbs

Defined Under Namespace

Modules: Source Classes: Options

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Internal::Type::BaseModel

==, #==, #[], coerce, #deconstruct_keys, #deep_to_h, dump, fields, hash, #hash, inherited, inspect, #inspect, known_fields, optional, recursively_to_h, required, #to_h, #to_json, #to_s, to_sorbet_type, #to_yaml

Methods included from Internal::Type::Converter

#coerce, coerce, #dump, dump, #inspect, inspect, meta_info, new_coerce_state, type_info

Methods included from Internal::Util::SorbetRuntimeSupport

#const_missing, #define_sorbet_constant!, #sorbet_constant_defined?, #to_sorbet_type, to_sorbet_type

Constructor Details

#initialize(max_urls: nil, regex: nil) ⇒ Object

Limits and filters for the sitemap URLs. A sitemap batch scrapes exactly those URLs and never follows links off them, so there is no crawl depth here.

Parameters:

  • max_urls (Integer) (defaults to: nil) —

    Maximum pages to fetch. Unused reserved credits are refunded. Maximum 25000.

  • regex (String) (defaults to: nil) —

    RE2 pattern; only sitemap URLs matching it are scraped.




# File 'lib/context_dev/models/batch_submit_params.rb', line 1789

Instance Attribute Details

#format_ ⇒ Symbol, :html

Return page content as HTML.

Parameters:

  • value (:html)

Returns:

  • (Symbol, :html) 


1775
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1775

required :format_, const: :html, api_name: :format

#options ⇒ ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::HTML::Options?

Options for HTML output.

Returns:



1787
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1787

optional :options, -> { ContextDev::BatchSubmitParams::Input::Crawl::Data::HTML::Options }

#source ⇒ ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::HTML::Source::StartURL, ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::HTML::Source::Sitemap

How to find pages to crawl.

Parameters:

  • value (ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::HTML::source)

Returns:



1781
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1781

required :source, union: -> { ContextDev::BatchSubmitParams::Input::Crawl::Data::HTML::Source }

Instance Method Details

#to_hash ⇒ {

Returns:

  • ({) 


2101
 
# File 'sig/context_dev/models/batch_submit_params.rbs', line 2101

def to_hash: -> {