Class: ContextDev::Models::WebExtractParams

Inherits:
Internal::Type::BaseModel show all
Extended by:
Internal::Type::RequestParameters::Converter
Includes:
Internal::Type::RequestParameters
Defined in:
lib/context_dev/models/web_extract_params.rb,
sig/context_dev/models/web_extract_params.rbs

Overview

See Also:

  • ContextDev::Resources::Web#extract

Defined Under Namespace

Modules: Action Classes: Pdf

Constant Summary

Constants included from Internal::Type::RequestParameters

Internal::Type::RequestParameters::ContextDev

Instance Attribute Summary collapse

Attributes included from Internal::Type::RequestParameters

#request_options

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Internal::Type::RequestParameters::Converter

dump_request

Methods included from Internal::Type::RequestParameters

included

Methods inherited from Internal::Type::BaseModel

==, #==, #[], coerce, #deconstruct_keys, #deep_to_h, dump, fields, hash, #hash, inherited, inspect, #inspect, known_fields, optional, recursively_to_h, required, #to_h, #to_json, #to_s, to_sorbet_type, #to_yaml

Methods included from Internal::Type::Converter

#coerce, coerce, #dump, dump, #inspect, inspect, meta_info, new_coerce_state, type_info

Methods included from Internal::Util::SorbetRuntimeSupport

#const_missing, #define_sorbet_constant!, #sorbet_constant_defined?, #to_sorbet_type, to_sorbet_type

Constructor Details

#initialize(schema:, url:, actions: nil, fact_check: nil, follow_subdomains: nil, include_frames: nil, instructions: nil, max_age_ms: nil, max_depth: nil, max_pages: nil, pdf: nil, settle_animations: nil, stop_after_ms: nil, tags: nil, timeout_ms: nil, wait_for_ms: nil, request_options: {}) ⇒ Object

Some parameter documentations has been truncated, see ContextDev::Models::WebExtractParams for more details.

Parameters:

  • schema (Hash{Symbol=>Object}) —

    JSON Schema for the returned data object. Image fields such as image_urls or `

  • url (String) —

    The starting website URL to crawl and extract from. Must include http:// or http

  • actions (Array<ContextDev::Models::WebExtractParams::Action::Wait, ContextDev::Models::WebExtractParams::Action::Perform, ContextDev::Models::WebExtractParams::Action::Scroll>) (defaults to: nil) —

    Optional browser actions executed in order on the requested page after it loads,

  • fact_check (Boolean) (defaults to: nil) —

    When true, every returned value must be grounded in facts stated on the page; fi

  • follow_subdomains (Boolean) (defaults to: nil) —

    When true, follow links on subdomains of the starting URL's domain.

  • include_frames (Boolean) (defaults to: nil) —

    When true, iframe contents are included in Markdown before extraction.

  • instructions (String) (defaults to: nil) —

    Optional extraction guidance, such as which facts to prioritize or how to interp

  • max_age_ms (Integer) (defaults to: nil) —

    Return cached scrape results if a prior scrape for the same parameters is younge

  • max_depth (Integer) (defaults to: nil) —

    Optional maximum link depth from the starting URL (0 = only the starting page).

  • max_pages (Integer) (defaults to: nil) —

    Maximum number of pages to analyze for extraction. Hard cap: 50. Defaults to 5.

  • pdf (ContextDev::Models::WebExtractParams::Pdf) (defaults to: nil)
  • settle_animations (Boolean) (defaults to: nil) —

    When true, waits briefly for CSS and transition animations to settle before extr

  • stop_after_ms (Integer) (defaults to: nil) —

    Soft time budget for the crawl in milliseconds. Min: 10000 (10s). Max: 110000 (1

  • tags (Array<String>) (defaults to: nil) —

    Optional tags for tracking usage. Up to 20 tags, each 1 to 50 characters.

  • timeout_ms (Integer) (defaults to: nil) —

    Optional timeout in milliseconds for the request. If the request takes longer th

  • wait_for_ms (Integer) (defaults to: nil) —

    Optional browser wait time in milliseconds after initial page load for each craw

  • request_options (ContextDev::RequestOptions, Hash{Symbol=>Object}) (defaults to: {}) 



# File 'lib/context_dev/models/web_extract_params.rb', line 128

Instance Attribute Details

#actions ⇒ Array<ContextDev::Models::WebExtractParams::Action::Wait, ContextDev::Models::WebExtractParams::Action::Perform, ContextDev::Models::WebExtractParams::Action::Scroll>?

Optional browser actions executed in order on the requested page after it loads, before links are discovered or additional pages are crawled. Requires a paid plan. When actions are provided and stopAfterMs is omitted, the crawl budget defaults to 110000 ms.

Returns:



34
 
# File 'lib/context_dev/models/web_extract_params.rb', line 34

optional :actions, -> { ContextDev::Internal::Type::ArrayOf[union: ContextDev::WebExtractParams::Action] }

#fact_check ⇒ Boolean?

When true, every returned value must be grounded in facts stated on the page; fields that cannot be supported by the page are returned as null/empty. When false (default), the model may make reasonable inferences and derivations from the page content (e.g. ideal customer, competitor analysis, recommendations) while keeping verifiable specifics (names, quotes, URLs, dates, metrics) faithful to the source.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


45
 
# File 'lib/context_dev/models/web_extract_params.rb', line 45

optional :fact_check, ContextDev::Internal::Type::Boolean, api_name: :factCheck

#follow_subdomains ⇒ Boolean?

When true, follow links on subdomains of the starting URL's domain.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


51
 
# File 'lib/context_dev/models/web_extract_params.rb', line 51

optional :follow_subdomains, ContextDev::Internal::Type::Boolean, api_name: :followSubdomains

#include_frames ⇒ Boolean?

When true, iframe contents are included in Markdown before extraction.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


57
 
# File 'lib/context_dev/models/web_extract_params.rb', line 57

optional :include_frames, ContextDev::Internal::Type::Boolean, api_name: :includeFrames

#instructions ⇒ String?

Optional extraction guidance, such as which facts to prioritize or how to interpret fields in the schema.

Parameters:

  • (String)

Returns:

  • (String, nil) 


64
 
# File 'lib/context_dev/models/web_extract_params.rb', line 64

optional :instructions, String

#max_age_ms ⇒ Integer?

Return cached scrape results if a prior scrape for the same parameters is younger than this many milliseconds. Defaults to 7 days (604800000 ms).

Parameters:

  • (Integer)

Returns:

  • (Integer, nil) 


71
 
# File 'lib/context_dev/models/web_extract_params.rb', line 71

optional :max_age_ms, Integer, api_name: :maxAgeMs

#max_depth ⇒ Integer?

Optional maximum link depth from the starting URL (0 = only the starting page). If omitted, there is no crawl depth limit.

Parameters:

  • (Integer)

Returns:

  • (Integer, nil) 


78
 
# File 'lib/context_dev/models/web_extract_params.rb', line 78

optional :max_depth, Integer, api_name: :maxDepth

#max_pages ⇒ Integer?

Maximum number of pages to analyze for extraction. Hard cap: 50. Defaults to 5.

Parameters:

  • (Integer)

Returns:

  • (Integer, nil) 


84
 
# File 'lib/context_dev/models/web_extract_params.rb', line 84

optional :max_pages, Integer, api_name: :maxPages

#pdf ⇒ ContextDev::Models::WebExtractParams::Pdf?

Returns:



89
 
# File 'lib/context_dev/models/web_extract_params.rb', line 89

optional :pdf, -> { ContextDev::WebExtractParams::Pdf }

#schema ⇒ Hash{Symbol=>Object}

JSON Schema for the returned data object. Image fields such as image_urls or product_photos automatically make page image references available to extraction, so product data and photos can be returned in one call. TypeScript Zod users can pass a JSON Schema generated from a Zod object; Python users can pass the equivalent JSON Schema object.

Parameters:

  • value (::Hash[Symbol, top])

Returns:

  • (Hash{Symbol=>Object}) 


18
 
# File 'lib/context_dev/models/web_extract_params.rb', line 18

required :schema, ContextDev::Internal::Type::HashOf[ContextDev::Internal::Type::Unknown]

#settle_animations ⇒ Boolean?

When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.

Parameters:

  • (Boolean)

Returns:

  • (Boolean, nil) 


97
 
# File 'lib/context_dev/models/web_extract_params.rb', line 97

optional :settle_animations, ContextDev::Internal::Type::Boolean, api_name: :settleAnimations

#stop_after_ms ⇒ Integer?

Soft time budget for the crawl in milliseconds. Min: 10000 (10s). Max: 110000 (110s). Defaults to 80000 (80s), or 110000 (110s) when browser actions are provided.

Parameters:

  • (Integer)

Returns:

  • (Integer, nil) 


105
 
# File 'lib/context_dev/models/web_extract_params.rb', line 105

optional :stop_after_ms, Integer, api_name: :stopAfterMs

#tags ⇒ Array<String>?

Optional tags for tracking usage. Up to 20 tags, each 1 to 50 characters.

Parameters:

  • (::Array[String])

Returns:

  • (Array<String>, nil) 


111
 
# File 'lib/context_dev/models/web_extract_params.rb', line 111

optional :tags, ContextDev::Internal::Type::ArrayOf[String]

#timeout_ms ⇒ Integer?

Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).

Parameters:

  • (Integer)

Returns:

  • (Integer, nil) 


119
 
# File 'lib/context_dev/models/web_extract_params.rb', line 119

optional :timeout_ms, Integer, api_name: :timeoutMS

#url ⇒ String

The starting website URL to crawl and extract from. Must include http:// or https://.

Parameters:

  • value (String)

Returns:

  • (String) 


25
 
# File 'lib/context_dev/models/web_extract_params.rb', line 25

required :url, String

#wait_for_ms ⇒ Integer?

Optional browser wait time in milliseconds after initial page load for each crawled page.

Parameters:

  • (Integer)

Returns:

  • (Integer, nil) 


126
 
# File 'lib/context_dev/models/web_extract_params.rb', line 126

optional :wait_for_ms, Integer, api_name: :waitForMs

Class Method Details

.values ⇒ Array<Symbol>

Returns:

  • (Array<Symbol>) 



# File 'lib/context_dev/models/web_extract_params.rb', line 306

.variants ⇒ Array(ContextDev::Models::WebExtractParams::Action::Wait, ContextDev::Models::WebExtractParams::Action::Perform, ContextDev::Models::WebExtractParams::Action::Scroll)

Returns:




# File 'lib/context_dev/models/web_extract_params.rb', line 311

Instance Method Details

#to_hash ⇒ {

Returns:

  • ({) 


112
 
# File 'sig/context_dev/models/web_extract_params.rbs', line 112

def to_hash: -> {