Module: Ghostcrawl::ResponseHelper Private

Defined in:
lib/ghostcrawl/client.rb

This module is part of a private API. You should avoid using this module if possible, as it may be removed or be changed in the future.

Class Method Summary collapse

Class Method Details

.parsable_to_hash(value) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Converts a typed Kiota Parsable into a plain Hash by serializing it via its own serialize and parsing the emitted JSON, then merging additional_data. Falls back to additional_data / to_h / the value itself on any failure.



281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
 
# File 'lib/ghostcrawl/client.rb', line 281

def self.parsable_to_hash(value)
  serialized = serialize_parsable(value)
  if serialized.is_a?(Hash)
    extra = (value.additional_data if value.respond_to?(:additional_data)) || {}
    merged = serialized.merge(extra.transform_keys(&:to_s))
    return merged.transform_values { |v| to_hash(v) }
  end

  if value.respond_to?(:additional_data) && value.additional_data && !value.additional_data.empty?
    value.additional_data.transform_values { |v| to_hash(v) }
  elsif value.respond_to?(:to_h)
    value.to_h.transform_values { |v| to_hash(v) }
  elsif value.respond_to?(:additional_data) && value.additional_data
    value.additional_data.transform_values { |v| to_hash(v) }
  else
    value
  end
end

.raise_on_result_error!(hash) ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Inspects a decoded HTTP-200 response hash for a RESULT-channel failure (the target page could not be scraped) and raises ScrapeError when one is present. This is the reliable, highest-value error path: the body is always available here (unlike the dropped problem+json body on non-2xx).

A failure is signalled by any of:

* a +result_error+ Hash carrying a +code+
* +ok+ explicitly +false+ (always a failure, even with no code)
* a top-level +code+ that is a known RESULT-channel code

A genuinely OK hash (ok: true, no ok key, or no error code) is returned untouched and never raises.

Parameters:

  • hash (Hash) —

    the decoded response

Returns:

  • (Hash) —

    the same hash, when it is not a failure

Raises:



400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
 
# File 'lib/ghostcrawl/client.rb', line 400

def self.raise_on_result_error!(hash)
  return hash unless hash.is_a?(Hash)

  # Descend into a `results` envelope (scrape/extract wrap per-URL results) —
  # the target failure lives on the INNER result, not the envelope top level.
  inner = hash["results"]
  if inner.is_a?(Array)
    inner.each { |item| raise_on_result_error!(item) }
    return hash
  end

  result_error = hash["result_error"]
  result_error = nil unless result_error.is_a?(Hash)
  top_code = hash["code"]
  ok_false = hash["ok"] == false

  # Pull the code: result_error wins, then a top-level RESULT-channel code.
  code = nil
  code = result_error["code"] if result_error
  code ||= top_code if Ghostcrawl::ErrorCodes.result_channel?(top_code)

  # The flat markdown-build envelope reports a target failure ONLY via
  # status="failed" (no ok/result_error) — don't count it as a success.
  status_failed = hash["status"] == "failed"
  code ||= top_code if status_failed && top_code.is_a?(String)

  # Only raise when there is a concrete result-channel failure signal.
  return hash unless code || ok_false || status_failed

  # ok: false with no usable code -> treat as empty/unusable content.
  code ||= Ghostcrawl::ErrorCodes::EMPTY_CONTENT

  retryable =
    if result_error && result_error.key?("retryable")
      result_error["retryable"]
    else
      Ghostcrawl::ErrorCodes::RETRYABLE.fetch(code, false)
    end

  target_status = nil
  target_status = result_error["target_status"] if result_error
  target_status ||= hash["target_status"]  # flat markdown-envelope path
  reason = result_error && result_error["reason"]

  msg = "scrape failed (#{code})"
  msg += ": #{reason}" if reason && !reason.to_s.empty?
  msg += " (target HTTP #{target_status})" if target_status

  raise Ghostcrawl::ScrapeError.new(
    msg,
    status_code: 200,
    body: nil,
    code: code,
    retryable: retryable,
    request_id: hash["request_id"],
    target_status: target_status
  )
end

.serialize_parsable(value) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Serializes a Parsable to a Ruby Hash using the pinned Kiota JSON writer. Returns the parsed Hash, or nil when the model isn't a serializable Parsable or serialization/parse fails (caller then falls back).



304
305
306
307
308
309
310
311
312
313
314
315
316
317
 
# File 'lib/ghostcrawl/client.rb', line 304

def self.serialize_parsable(value)
  return nil unless value.respond_to?(:serialize)

  writer = MicrosoftKiotaSerializationJson::JsonSerializationWriter.new
  value.serialize(writer)
  content = writer.get_serialized_content
  json = content.is_a?(String) ? content : content.read
  return nil if json.nil? || json.empty?

  parsed = JSON.parse(json)
  parsed.is_a?(Hash) ? parsed : nil
rescue StandardError
  nil
end

.to_hash(value) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Converts any Kiota response value to a plain Hash or Array. The Kiota Faraday adapter returns Fibers for async responses; call .resume to execute the request synchronously and get the actual response value.



237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
 
# File 'lib/ghostcrawl/client.rb', line 237

def self.to_hash(value)
  # Resolve Fibers (the Kiota Faraday adapter returns a Fiber for every
  # request; .resume is what actually executes the HTTP call). Any non-2xx
  # surfaces here as a raw transport exception — translate it into a typed
  # Ghostcrawl error so the documented rescue contract works.
  if value.is_a?(Fiber)
    begin
      value = value.resume
    rescue Ghostcrawl::GhostcrawlError
      raise
    rescue StandardError => e
      Ghostcrawl.raise_translated(e)
    end
  end

  case value
  when Hash
    value.transform_values { |v| to_hash(v) }
  when Array
    value.map { |v| to_hash(v) }
  when NilClass
    {}
  else
    # Typed Parsable (Kiota-generated model). Kiota models keep their spec
    # fields in typed instance variables, NOT in +additional_data+ (which is
    # only the overflow bucket for unmapped keys). Reading +additional_data+
    # alone therefore returns +{}+ for a fully-typed response (e.g.
    # MapResponse{@links,@success}, WebhookListResponse{@items,@total}),
    # silently dropping the real payload.
    #
    # Recover the typed fields by round-tripping the model through its own
    # +serialize+ (the Parsable contract) into a JSON writer and parsing the
    # result back to a plain Hash. Then overlay any +additional_data+ (unmapped
    # keys the server sent that the model didn't declare). Degrade gracefully:
    # a model whose +serialize+ raises (e.g. an unresolved composed-type member)
    # falls back to the prior additional_data/to_h behavior so nothing regresses.
    parsable_to_hash(value)
  end
end

.void_request!(adapter, request_info) ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Executes a request that returns NO response body (an HTTP 204, or any DELETE whose spec maps the success response to void) and returns a plain Hash.

Why this exists — two upstream defects in the pinned Kiota Ruby runtime that make the generated void-DELETE builders unusable:

1. The generated +delete+ for a void response calls
 +send_async(request_info, nil, ...)+, and the Faraday adapter hard-raises
 +"factory cannot be null"+ BEFORE the request is ever sent — so the DELETE
 never reaches the server and the caller gets a bare +StandardError+
 (not a {GhostcrawlError}).
2. For void DELETEs the adapter DOES send maps to +Binary+, the response is an
 empty 204 body, and +get_root_parse_node+ then feeds +""+ to the JSON parser,
 raising +JSON::ParserError+ on a perfectly successful delete.

This helper reuses the adapter's OWN public request pipeline (+convert_to_native_request_async+ applies base-url + auth, run_request sends it) but skips body deserialization entirely: a 2xx returns {} (or the decoded JSON when the server did send a body, e.g. {"deleted":true}), and a

=400 is translated into the documented typed GhostcrawlError hierarchy.

Parameters:

  • adapter (MicrosoftKiotaFaraday::FaradayRequestAdapter)
  • request_info (MicrosoftKiotaAbstractions::RequestInformation)

Returns:

  • (Hash) —

    {} on an empty 2xx, or the decoded body when one is present



344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
 
# File 'lib/ghostcrawl/client.rb', line 344

def self.void_request!(adapter, request_info)
  # Replicate the first half of the adapter's send_async by hand — apply the
  # base URL, run the bearer-auth Fiber (which mutates request_info's headers),
  # then build the native Faraday request. We deliberately do NOT call the
  # adapter's convert_to_native_request_async helper: its body uses a bare
  # +return+ inside a Fiber, which raises LocalJumpError("unexpected return")
  # when resumed under Ruby 3.x.
  request_info.path_parameters["baseurl"] = adapter.get_base_url
  # authenticate_request returns a Fiber only when the Authorization header is
  # not already present; it returns nil otherwise. Guard the resume.
  auth_fiber = adapter.authentication_provider.authenticate_request(request_info)
  auth_fiber.resume if auth_fiber.respond_to?(:resume)
  request = adapter.get_request_from_request_info(request_info)
  response = adapter.client.run_request(
    request.http_method, request.path, request.body, request.headers
  )

  status = response.status
  if status >= 400
    # Reuse the typed-error translation: synthesize a message carrying the
    # ":<status>" token raise_translated keys off, plus any body text.
    body = response.body.to_s
    err = MicrosoftKiotaAbstractions::ApiError.new(
      "The server returned an unexpected status code:#{status}" \
      "#{body.empty? ? '' : " #{body}"}"
    )
    Ghostcrawl.raise_translated(err)
  end

  body = response.body.to_s
  return {} if body.strip.empty?

  parsed = (JSON.parse(body) rescue nil)
  parsed.is_a?(Hash) ? parsed.transform_values { |v| to_hash(v) } : {}
rescue Ghostcrawl::GhostcrawlError
  raise
rescue StandardError => e
  Ghostcrawl.raise_translated(e)
end