Class: ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::Markdown::Options

Inherits:
Internal::Type::BaseModel show all
Defined in:
lib/context_dev/models/batch_submit_params.rb,
sig/context_dev/models/batch_submit_params.rbs

Overview

See Also:

  • ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::Markdown#options

Defined Under Namespace

Modules: Country Classes: Pdf

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Internal::Type::BaseModel

==, #==, #[], coerce, #deconstruct_keys, #deep_to_h, dump, fields, hash, #hash, inherited, inspect, #inspect, known_fields, optional, recursively_to_h, required, #to_h, #to_json, #to_s, to_sorbet_type, #to_yaml

Methods included from Internal::Type::Converter

#coerce, coerce, #dump, dump, #inspect, inspect, meta_info, new_coerce_state, type_info

Methods included from Internal::Util::SorbetRuntimeSupport

#const_missing, #define_sorbet_constant!, #sorbet_constant_defined?, #to_sorbet_type, to_sorbet_type

Constructor Details

#initialize(country: nil, exclude_selectors: nil, include_images: nil, include_links: nil, include_selectors: nil, max_age_ms: nil, pdf: nil, settle_animations: nil, shorten_base64_images: nil, use_main_content_only: nil, wait_for_ms: nil) ⇒ Object

Some parameter documentations has been truncated, see ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::Markdown::Options for more details.

Options for Markdown output.

Parameters:

  • country (Symbol, ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::Markdown::Options::Country) (defaults to: nil) —

    Fetch the target page through a residential proxy in this country (ISO 3166-1 al

  • exclude_selectors (Array<String>, nil) (defaults to: nil) —

    Remove elements matching these CSS selectors. Applied after includeSelectors,

  • include_images (Boolean) (defaults to: nil) —

    Include image references in the Markdown.

  • include_links (Boolean) (defaults to: nil) —

    Include links in the Markdown.

  • include_selectors (Array<String>, nil) (defaults to: nil) —

    Keep only the subtrees matching these CSS selectors. Filtered pages are always f

  • max_age_ms (Integer, nil) (defaults to: nil) —

    Return a cached result if a prior scrape for the same parameters exists and is y

  • pdf (ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::Markdown::Options::Pdf) (defaults to: nil) —

    PDF parsing controls. Use start/end to limit text extraction and embedded-image

  • settle_animations (Boolean) (defaults to: nil) —

    Wait briefly for CSS and transition animations to settle before extraction, on p

  • shorten_base64_images (Boolean) (defaults to: nil) —

    Shorten inline base64 image data.

  • use_main_content_only (Boolean) (defaults to: nil) —

    Return the main content without navigation or footers.

  • wait_for_ms (Integer) (defaults to: nil) —

    How long to wait after initial page load, in milliseconds. 0 waits 500 ms.




# File 'lib/context_dev/models/batch_submit_params.rb', line 1402

Instance Attribute Details

#country ⇒ Symbol, ...

Fetch the target page through a residential proxy in this country (ISO 3166-1 alpha-2).

Returns:



1323
1324
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1323

optional :country,
enum: -> { ContextDev::BatchSubmitParams::Input::Crawl::Data::Markdown::Options::Country }

#exclude_selectors ⇒ Array<String>?

Remove elements matching these CSS selectors. Applied after includeSelectors, so an element matching both is removed.

Parameters:

  • value (::Array[String], nil)

Returns:

  • (Array<String>, nil) 


1331
1332
1333
1334
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1331

optional :exclude_selectors,
ContextDev::Internal::Type::ArrayOf[String],
api_name: :excludeSelectors,
nil?: true

#include_images ⇒ Boolean?

Include image references in the Markdown.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


1340
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1340

optional :include_images, ContextDev::Internal::Type::Boolean, api_name: :includeImages

#include_links ⇒ Boolean?

Include links in the Markdown.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


1346
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1346

optional :include_links, ContextDev::Internal::Type::Boolean, api_name: :includeLinks

#include_selectors ⇒ Array<String>?

Keep only the subtrees matching these CSS selectors. Filtered pages are always fetched fresh, ignoring maxAgeMs.

Parameters:

  • value (::Array[String], nil)

Returns:

  • (Array<String>, nil) 


1353
1354
1355
1356
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1353

optional :include_selectors,
ContextDev::Internal::Type::ArrayOf[String],
api_name: :includeSelectors,
nil?: true

#max_age_ms ⇒ Integer?

Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh.

Parameters:

  • value (Integer, nil)

Returns:

  • (Integer, nil) 


1364
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1364

optional :max_age_ms, Integer, api_name: :maxAgeMs, nil?: true

#pdf ⇒ ContextDev::Models::BatchSubmitParams::Input::Crawl::Data::Markdown::Options::Pdf?

PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range.

Returns:



1371
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1371

optional :pdf, -> { ContextDev::BatchSubmitParams::Input::Crawl::Data::Markdown::Options::Pdf }

#settle_animations ⇒ Boolean?

Wait briefly for CSS and transition animations to settle before extraction, on pages that render in a browser.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


1378
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1378

optional :settle_animations, ContextDev::Internal::Type::Boolean, api_name: :settleAnimations

#shorten_base64_images ⇒ Boolean?

Shorten inline base64 image data.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


1384
1385
1386
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1384

optional :shorten_base64_images,
ContextDev::Internal::Type::Boolean,
api_name: :shortenBase64Images

#use_main_content_only ⇒ Boolean?

Return the main content without navigation or footers.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


1392
1393
1394
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1392

optional :use_main_content_only,
ContextDev::Internal::Type::Boolean,
api_name: :useMainContentOnly

#wait_for_ms ⇒ Integer?

How long to wait after initial page load, in milliseconds. 0 waits 500 ms.

Parameters:

  • (Integer)

Returns:

  • (Integer, nil) 


1400
 
# File 'lib/context_dev/models/batch_submit_params.rb', line 1400

optional :wait_for_ms, Integer, api_name: :waitForMs

Class Method Details

.values ⇒ Array<Symbol>

Returns:

  • (Array<Symbol>) 



# File 'lib/context_dev/models/batch_submit_params.rb', line 1643

Instance Method Details

#to_hash ⇒ {

Returns:

  • ({) 


1578
 
# File 'sig/context_dev/models/batch_submit_params.rbs', line 1578

def to_hash: -> {