Class: ContextDev::Models::WebWebScrapeMdParams

Inherits:

Internal::Type::BaseModel

• Object
• Internal::Type::BaseModel
• ContextDev::Models::WebWebScrapeMdParams

Extended by:

Internal::Type::RequestParameters::Converter

Includes:

Internal::Type::RequestParameters

Defined in:

lib/context_dev/models/web_web_scrape_md_params.rb,
sig/context_dev/models/web_web_scrape_md_params.rbs

Overview

See Also:

• ContextDev::Resources::Web#web_scrape_md

Defined Under Namespace

Modules: Action, Country, Zdr Classes: Pdf

Constant Summary

Constants included from Internal::Type::RequestParameters

Internal::Type::RequestParameters::ContextDev

Instance Attribute Summary

• #actions ⇒ Array<ContextDev::Models::WebWebScrapeMdParams::Action::Wait, ContextDev::Models::WebWebScrapeMdParams::Action::Perform>^?

  Optional browser actions executed in array order after the page loads and before content is captured.

• #country ⇒ Symbol, ...

  Fetch the target page through a residential proxy in this country (ISO 3166-1 alpha-2).

• #exclude_selectors ⇒ Array<String>^?

  CSS selectors to remove before conversion to Markdown.

• #headers ⇒ Hash{Symbol=>String}^?

  Optional outbound HTTP headers forwarded only to the target URL, sent as deep-object query params such as headers=value.

• #include_frames ⇒ Boolean^?

  When true, the contents of iframes are rendered to Markdown.

• #include_html ⇒ Boolean^?

  When true, the response also includes an html field with the page HTML the Markdown was converted from — the same body the Scrape HTML endpoint returns for the equivalent request.

• #include_images ⇒ Boolean^?

  Include image references in Markdown output.

• #include_links ⇒ Boolean^?

  Preserve hyperlinks in Markdown output.

• #include_selectors ⇒ Array<String>^?

  CSS selectors.

• #max_age_ms ⇒ Integer^?

  Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds.

• #pdf ⇒ ContextDev::Models::WebWebScrapeMdParams::Pdf^?

  PDF parsing controls.

• #settle_animations ⇒ Boolean^?

  When true, waits briefly for CSS and transition animations to settle before converting to Markdown.

• #shorten_base64_images ⇒ Boolean^?

  Shorten base64-encoded image data in the Markdown output.

• #tags ⇒ Array<String>^?

  Optional comma-separated caller-defined tags for tracking this request.

• #timeout_ms ⇒ Integer^?

  Optional timeout in milliseconds for the request.

• #url ⇒ String

  Full URL to scrape into LLM usable Markdown (must include http:// or https:// protocol).

• #use_main_content_only ⇒ Boolean^?

  Extract only the main content of the page, excluding headers, footers, sidebars, and navigation.

• #wait_for_ms ⇒ Integer^?

  Optional browser wait time in milliseconds after initial page load before converting the page to Markdown.

• #zdr ⇒ Symbol, ...

  Set to enabled to bypass shared caches and omit request and response content from retained usage logs.

Attributes included from Internal::Type::RequestParameters

#request_options

Class Method Summary

• .values ⇒ Array<Symbol>
• .variants ⇒ Array(ContextDev::Models::WebWebScrapeMdParams::Action::Wait, ContextDev::Models::WebWebScrapeMdParams::Action::Perform)

Instance Method Summary

• #initialize(end_: nil, ocr: nil, should_parse: nil, start: nil) ⇒ Object constructor

  Some parameter documentations has been truncated, see Pdf for more details.

• #to_hash ⇒ {

Methods included from Internal::Type::RequestParameters::Converter

dump_request

Methods included from Internal::Type::RequestParameters

included

Methods inherited from Internal::Type::BaseModel

==, #==, #[], coerce, #deconstruct_keys, #deep_to_h, dump, fields, hash, #hash, inherited, inspect, #inspect, known_fields, optional, recursively_to_h, required, #to_h, #to_json, #to_s, to_sorbet_type, #to_yaml

Methods included from Internal::Type::Converter

#coerce, coerce, #dump, dump, #inspect, inspect, meta_info, new_coerce_state, type_info

Methods included from Internal::Util::SorbetRuntimeSupport

#const_missing, #define_sorbet_constant!, #sorbet_constant_defined?, #to_sorbet_type, to_sorbet_type

Constructor Details

#initialize(end_: nil, ocr: nil, should_parse: nil, start: nil) ⇒ Object

Some parameter documentations has been truncated, see Pdf for more details.

PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range.

Parameters:

• end_ (Integer) (defaults to: nil) —

  Last 1-based PDF page to parse. When omitted, parsing ends at the last page. Mus

• ocr (Boolean) (defaults to: nil) —

  When true, OCR the selected PDF pages that have no usable text layer (scans), re

• should_parse (Boolean) (defaults to: nil) —

  When true, PDF URLs are fetched and parsed. When false, PDF URLs are skipped and

• start (Integer) (defaults to: nil) —

  First 1-based PDF page to parse. When omitted, parsing starts at the first page.

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 152

Instance Attribute Details

#actions ⇒ Array<ContextDev::Models::WebWebScrapeMdParams::Action::Wait, ContextDev::Models::WebWebScrapeMdParams::Action::Perform>^?

Optional browser actions executed in array order after the page loads and before content is captured. Requires a paid plan. Send a JSON array in the query parameter. Maximum: 5 actions.

Parameters:

• value (::Array[ContextDev::Models::WebWebScrapeMdParams::action], nil)

Returns:

• (Array<ContextDev::Models::WebWebScrapeMdParams::Action::Wait, ContextDev::Models::WebWebScrapeMdParams::Action::Perform>, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 23

optional :actions,
-> { ContextDev::Internal::Type::ArrayOf[union: ContextDev::WebWebScrapeMdParams::Action] },
nil?: true

#country ⇒ Symbol, ...

Fetch the target page through a residential proxy in this country (ISO 3166-1 alpha-2).

Returns:

• (Symbol, ContextDev::Models::WebWebScrapeMdParams::Country, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 32

optional :country, enum: -> { ContextDev::WebWebScrapeMdParams::Country }

#exclude_selectors ⇒ Array<String>^?

CSS selectors to remove before conversion to Markdown. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Examples: "nav", "footer", ".ad-banner", "[aria-hidden=true]".

Parameters:

• value (::Array[String], nil)

Returns:

• (Array<String>, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 40

optional :exclude_selectors, ContextDev::Internal::Type::ArrayOf[String], nil?: true

#headers ⇒ Hash{Symbol=>String}^?

Optional outbound HTTP headers forwarded only to the target URL, sent as deep-object query params such as headers=value. When provided, caching is bypassed: the result is neither read from nor written to cache.

Parameters:

• (::Hash[Symbol, String])

Returns:

• (Hash{Symbol=>String}, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 48

optional :headers, ContextDev::Internal::Type::HashOf[String]

#include_frames ⇒ Boolean^?

When true, the contents of iframes are rendered to Markdown.

Parameters:

• (Boolean)

Returns:

• (Boolean, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 54

optional :include_frames, ContextDev::Internal::Type::Boolean

#include_html ⇒ Boolean^?

When true, the response also includes an html field with the page HTML the Markdown was converted from — the same body the Scrape HTML endpoint returns for the equivalent request.

Parameters:

• (Boolean)

Returns:

• (Boolean, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 62

optional :include_html, ContextDev::Internal::Type::Boolean

#include_images ⇒ Boolean^?

Include image references in Markdown output

Parameters:

• (Boolean)

Returns:

• (Boolean, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 68

optional :include_images, ContextDev::Internal::Type::Boolean

#include_links ⇒ Boolean^?

Preserve hyperlinks in Markdown output

Parameters:

• (Boolean)

Returns:

• (Boolean, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 74

optional :include_links, ContextDev::Internal::Type::Boolean

#include_selectors ⇒ Array<String>^?

CSS selectors. When provided, only matching HTML subtrees (and their descendants) are kept before conversion to Markdown. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]".

Parameters:

• value (::Array[String], nil)

Returns:

• (Array<String>, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 82

optional :include_selectors, ContextDev::Internal::Type::ArrayOf[String], nil?: true

#max_age_ms ⇒ Integer^?

Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh.

Parameters:

• value (Integer, nil)

Returns:

• (Integer, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 90

optional :max_age_ms, Integer, nil?: true

#pdf ⇒ ContextDev::Models::WebWebScrapeMdParams::Pdf^?

PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range.

Returns:

• (ContextDev::Models::WebWebScrapeMdParams::Pdf, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 97

optional :pdf, -> { ContextDev::WebWebScrapeMdParams::Pdf }

#settle_animations ⇒ Boolean^?

When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.

Parameters:

• (Boolean)

Returns:

• (Boolean, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 105

optional :settle_animations, ContextDev::Internal::Type::Boolean

#shorten_base64_images ⇒ Boolean^?

Shorten base64-encoded image data in the Markdown output

Parameters:

• (Boolean)

Returns:

• (Boolean, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 111

optional :shorten_base64_images, ContextDev::Internal::Type::Boolean

#tags ⇒ Array<String>^?

Optional comma-separated caller-defined tags for tracking this request. Tags are recorded on the request's usage log and can be used to filter usage on the dashboard usage page. Up to 20 tags, each 1-50 characters.

Parameters:

• (::Array[String])

Returns:

• (Array<String>, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 119

optional :tags, ContextDev::Internal::Type::ArrayOf[String]

#timeout_ms ⇒ Integer^?

Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).

Parameters:

• (Integer)

Returns:

• (Integer, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 127

optional :timeout_ms, Integer

#url ⇒ String

Full URL to scrape into LLM usable Markdown (must include http:// or https:// protocol)

Parameters:

• value (String)

Returns:

• (String)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 15

required :url, String

#use_main_content_only ⇒ Boolean^?

Extract only the main content of the page, excluding headers, footers, sidebars, and navigation

Parameters:

• (Boolean)

Returns:

• (Boolean, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 134

optional :use_main_content_only, ContextDev::Internal::Type::Boolean

#wait_for_ms ⇒ Integer^?

Optional browser wait time in milliseconds after initial page load before converting the page to Markdown. Min: 0. Max: 30000 (30 seconds).

Parameters:

• value (Integer, nil)

Returns:

• (Integer, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 141

optional :wait_for_ms, Integer, nil?: true

#zdr ⇒ Symbol, ...

Set to enabled to bypass shared caches and omit request and response content from retained usage logs. Requires zero data retention to be enabled for your organization (contact support@context.dev), otherwise the request fails with ZDR_NOT_ENABLED. Successful ZDR responses include X-Context-ZDR: true.

Returns:

• (Symbol, ContextDev::Models::WebWebScrapeMdParams::Zdr, nil)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 150

optional :zdr, enum: -> { ContextDev::WebWebScrapeMdParams::Zdr }

Class Method Details

.values ⇒ Array<Symbol>

Returns:

• (Array<Symbol>)

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 459

.variants ⇒ Array(ContextDev::Models::WebWebScrapeMdParams::Action::Wait, ContextDev::Models::WebWebScrapeMdParams::Action::Perform)

Returns:

• (Array(ContextDev::Models::WebWebScrapeMdParams::Action::Wait, ContextDev::Models::WebWebScrapeMdParams::Action::Perform))

# File 'lib/context_dev/models/web_web_scrape_md_params.rb', line 245

Instance Method Details

#to_hash ⇒ {

Returns:

• ({)

# File 'sig/context_dev/models/web_web_scrape_md_params.rbs', line 124

def to_hash: -> {