Module: BrightData

Defined in:
lib/brightdata.rb,
lib/brightdata/http.rb,
lib/brightdata/client.rb,
lib/brightdata/errors.rb,
lib/brightdata/result.rb,
lib/brightdata/version.rb,
lib/brightdata/datasets.rb,
lib/brightdata/snapshot.rb,
lib/brightdata/live_trace.rb,
lib/brightdata/linkedin/jobs.rb,
lib/brightdata/linkedin/posts.rb,
lib/brightdata/linkedin/people.rb,
lib/brightdata/linkedin/endpoint.rb,
lib/brightdata/linkedin/profiles.rb,
lib/brightdata/linkedin/companies.rb,
lib/brightdata/linkedin/namespace.rb,
lib/brightdata/linkedin/types/job.rb,
lib/brightdata/linkedin/types/post.rb,
lib/brightdata/linkedin/types/company.rb,
lib/brightdata/linkedin/types/profile.rb,
lib/brightdata/linkedin/types/job_url_input.rb,
lib/brightdata/linkedin/types/post_url_input.rb,
lib/brightdata/linkedin/types/company_url_input.rb,
lib/brightdata/linkedin/types/job_keyword_input.rb,
lib/brightdata/linkedin/types/profile_url_input.rb,
lib/brightdata/linkedin/types/discovered_profile.rb,
lib/brightdata/linkedin/types/people_discover_input.rb,
lib/brightdata/linkedin/types/post_company_url_input.rb,
lib/brightdata/linkedin/types/post_profile_url_input.rb

Overview

A typed, ergonomic Ruby client for Bright Data’s Datasets v3 scraper APIs.

Create a Client with an API token, then reach endpoints through namespaces such as ‘client.linkedin`. Every endpoint exposes two methods:

  • ‘#scrape(…)` runs synchronously and returns parsed, typed results. Bright Data caps synchronous scrapes at 60 seconds; if a job exceeds that, it raises ScrapeTimeoutError, which carries a resumable Snapshot.

  • ‘#trigger(…)` starts an asynchronous collection and returns a Snapshot you poll with `#wait`.

Results are immutable ‘Data` value objects exposing typed readers plus `#raw`, the full parsed response hash. All errors inherit from Error.

Examples:

Synchronous scrape

client = BrightData::Client.new(api_token: ENV.fetch("BRIGHTDATA_API_TOKEN"))
profiles = client.linkedin.profiles.scrape(urls: ["https://www.linkedin.com/in/example/"])
profiles.first.name #=> "Example Person"

Asynchronous trigger and poll

snapshot = client.linkedin.profiles.trigger(urls: ["https://www.linkedin.com/in/example/"])
result = snapshot.wait
result.payload if result.success?

Discovery by keyword (nil fields are omitted from the request)

query = BrightData::LinkedIn::Types::JobKeywordInput.new(
  location: "New York", keyword: "ruby",
  country: nil, time_range: nil, job_type: nil, experience_level: nil,
  remote: nil, company: nil, selective_search: nil,
  jobs_to_not_include: nil, location_radius: nil
)
jobs = client.linkedin.jobs.discover_by_keyword.scrape(queries: [query])

Recovering from a synchronous timeout

begin
  client.linkedin.profiles.scrape(urls: urls)
rescue BrightData::ScrapeTimeoutError => e
  e.snapshot.wait # fall back to async polling
end

Defined Under Namespace

Modules: Datasets, LinkedIn Classes: ArgumentError, AuthError, Client, ConfigurationError, Error, HTTP, HTTPError, LiveTrace, RateLimitError, ScrapeTimeoutError, ServerError, Snapshot, SnapshotFailedError

Constant Summary collapse

Result =

Result type re-exported from the ‘simple-result` gem.

See Also:

SimpleResult
VERSION =

Current gem version.

"0.1.0"