Class: Firecrawl::HttpClient Private

Inherits:
Object
  • Object
show all
Defined in:
lib/firecrawl/http_client.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Internal HTTP client for making authenticated requests to the Firecrawl API. Handles retry logic with exponential backoff.

Instance Method Summary collapse

Constructor Details

#initialize(api_key:, base_url:, timeout:, max_retries:, backoff_factor:) ⇒ HttpClient

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of HttpClient.



14
15
16
17
18
19
20
 
# File 'lib/firecrawl/http_client.rb', line 14

def initialize(api_key:, base_url:, timeout:, max_retries:, backoff_factor:)
  @api_key = api_key
  @base_url = base_url.chomp("/")
  @timeout = timeout
  @max_retries = max_retries
  @backoff_factor = backoff_factor
end

Instance Method Details

#delete(path) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Sends a DELETE request.



55
56
57
58
59
60
 
# File 'lib/firecrawl/http_client.rb', line 55

def delete(path)
  uri = URI("#{@base_url}#{path}")
  request = Net::HTTP::Delete.new(uri)
  request["Authorization"] = "Bearer #{@api_key}"
  execute_with_retry(uri, request)
end

#get(path) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Sends a GET request.



34
35
36
37
38
39
 
# File 'lib/firecrawl/http_client.rb', line 34

def get(path)
  uri = URI("#{@base_url}#{path}")
  request = Net::HTTP::Get.new(uri)
  request["Authorization"] = "Bearer #{@api_key}"
  execute_with_retry(uri, request)
end

#get_absolute(absolute_url) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Sends a GET request with a full URL (for following next-page cursors). Only sends the Authorization header if the URL matches the configured API origin.



43
44
45
46
47
48
49
50
51
52
 
# File 'lib/firecrawl/http_client.rb', line 43

def get_absolute(absolute_url)
  uri = URI(absolute_url)
  base_uri = URI(@base_url)
  unless uri.host == base_uri.host && uri.port == base_uri.port && uri.scheme == base_uri.scheme
    raise FirecrawlError, "Absolute URL origin (#{uri.scheme}://#{uri.host}:#{uri.port}) does not match API base URL origin (#{base_uri.scheme}://#{base_uri.host}:#{base_uri.port}). Refusing to send credentials."
  end
  request = Net::HTTP::Get.new(uri)
  request["Authorization"] = "Bearer #{@api_key}"
  execute_with_retry(uri, request)
end

#post(path, body, extra_headers: {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Sends a POST request with JSON body.



23
24
25
26
27
28
29
30
31
 
# File 'lib/firecrawl/http_client.rb', line 23

def post(path, body, extra_headers: {})
  uri = URI("#{@base_url}#{path}")
  request = Net::HTTP::Post.new(uri)
  request["Authorization"] = "Bearer #{@api_key}"
  request["Content-Type"] = "application/json"
  extra_headers.each { |k, v| request[k] = v }
  request.body = JSON.generate(body)
  execute_with_retry(uri, request)
end

#post_multipart(path, fields:, file_field:, filename:, content:, content_type: nil) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Sends a POST request with a multipart/form-data body.

Parameters:

  • path (String) —

    API path

  • fields (Hash{String=>String}) —

    additional form fields to include

  • file_field (String) —

    form field name for the file part (e.g. “file”)

  • filename (String) —

    filename to send with the file part

  • content (String) —

    raw bytes for the file part

  • content_type (String, nil) (defaults to: nil) —

    optional MIME type for the file part



70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/firecrawl/http_client.rb', line 70

def post_multipart(path, fields:, file_field:, filename:, content:, content_type: nil)
  uri = URI("#{@base_url}#{path}")
  boundary = "----FirecrawlBoundary#{SecureRandom.hex(16)}"
  body = build_multipart_body(boundary, fields, file_field, filename, content, content_type)

  builder = lambda do
    request = Net::HTTP::Post.new(uri)
    request["Authorization"] = "Bearer #{@api_key}"
    request["Content-Type"] = "multipart/form-data; boundary=#{boundary}"
    request.body = body
    request
  end

  execute_with_retry(uri, builder.call, request_builder: builder)
end