Class: Html2rss::RequestService::BotasaurusContract::Success

Inherits:
Object
  • Object
show all
Defined in:
lib/html2rss/request_service/botasaurus_contract.rb

Overview

Parsed OpenAPI ScrapeSuccess envelope (HTTP 200).

Constant Summary collapse

MAX_XHR_BODY_BYTES =

Per-body size cap for captured XHR JSON (defense-in-depth vs upstream).

500_000
MAX_XHR_AGGREGATE_BYTES =

Aggregate size cap across all captured XHR bodies.

2_000_000

Instance Method Summary collapse

Constructor Details

#initialize(payload:) ⇒ Success

Returns a new instance of Success.

Parameters:

  • payload (Hash{String => Object}) —

    parsed ScrapeSuccess object



58
59
60
61
62
 
# File 'lib/html2rss/request_service/botasaurus_contract.rb', line 58

def initialize(payload:)
  @payload = payload
  html
  diagnostics
end

Instance Method Details

#final_url ⇒ String?

Returns final URL reported by upstream.

Returns:

  • (String, nil) —

    final URL reported by upstream



87
 
# File 'lib/html2rss/request_service/botasaurus_contract.rb', line 87

def final_url = payload['final_url']

#headers ⇒ Hash{String => String}

Returns normalized response headers (null → {}).

Returns:

  • (Hash{String => String}) —

    normalized response headers (null → {})



73
74
75
76
77
78
 
# File 'lib/html2rss/request_service/botasaurus_contract.rb', line 73

def headers
  raw_headers = payload['headers']
  return {} unless raw_headers.is_a?(Hash) && raw_headers.any?

  raw_headers.to_h { |key, value| [key.to_s, value.to_s] }
end

#html ⇒ String

Returns rendered HTML body.

Returns:

  • (String) —

    rendered HTML body

Raises:



65
66
67
68
69
70
 
# File 'lib/html2rss/request_service/botasaurus_contract.rb', line 65

def html
  value = payload['html']
  raise BotasaurusServiceError, 'Botasaurus scrape success requires html' unless value.is_a?(String)

  value
end

#status ⇒ Integer?

Returns document status_code when present.

Returns:

  • (Integer, nil) —

    document status_code when present



81
82
83
84
 
# File 'lib/html2rss/request_service/botasaurus_contract.rb', line 81

def status
  status_code = payload['status_code']
  status_code.is_a?(Integer) ? status_code : nil
end

#transport_meta ⇒ Hash{String => Object}

Returns diagnostics plus success-only metadata_error (frozen).

Returns:

  • (Hash{String => Object}) —

    diagnostics plus success-only metadata_error (frozen)



90
91
92
93
94
 
# File 'lib/html2rss/request_service/botasaurus_contract.rb', line 90

def transport_meta
  meta = diagnostics.dup
  meta['metadata_error'] = metadata_error if metadata_error
  meta.freeze
end

#xhr_responses ⇒ Array<Hash{String => Object}>

Returns size-capped XHR/fetch captures (absent → []).

Returns:

  • (Array<Hash{String => Object}>) —

    size-capped XHR/fetch captures (absent → [])



97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/html2rss/request_service/botasaurus_contract.rb', line 97

def xhr_responses
  raw = payload['xhr_responses']
  return [] unless raw.is_a?(Array)

  aggregate_bytes = 0
  raw.filter_map do |entry|
    normalized = normalize_xhr_entry(entry)
    next unless normalized
    next unless (aggregate_bytes = advance_xhr_budget(normalized, aggregate_bytes))

    normalized
  end
end