Link Scraper API - Ruby Gem

Link Scraper is a simple tool for scraping web page links. It returns all the links on a web page.

Installation

Add this line to your application's Gemfile:

gem 'apiverve_linkscraper'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install apiverve_linkscraper

Getting Started

Get your API key at APIVerve

Basic Usage

require 'apiverve_linkscraper'

# Initialize the client
client = APIVerve::Linkscraper::Client.new(api_key: "YOUR_API_KEY")

# Make a request
response = client.execute({
  url: "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
  maxlinks: 20,
  includequery: false
})

# Print the response
puts response

Error Handling

begin
  response = client.execute({ url: "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html", maxlinks: 20, includequery: false })
  puts response["data"]
rescue APIVerve::Linkscraper::ValidationError => e
  puts "Validation error: #{e.errors.join(', ')}"
rescue APIVerve::Linkscraper::APIError => e
  puts "API error: #{e.message}"
  puts "Status code: #{e.status_code}"
end

Debug Mode

# Enable debug logging
client = APIVerve::Linkscraper::Client.new(
  api_key: "YOUR_API_KEY",
  debug: true
)

Example Response

{
  "status": "ok",
  "error": null,
  "data": {
    "url": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
    "linkCount": 16,
    "externalLinkCount": 13,
    "internalLinkCount": 3,
    "links": [
      {
        "text": "Documentation",
        "href": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/index.html",
        "external": false
      },
      {
        "text": "Amazon EC2 Instance Types Guide",
        "href": "https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-types.html",
        "external": true
      },
      {
        "text": "Amazon EC2 Auto Scaling",
        "href": "https://docs.aws.amazon.com/autoscaling/",
        "external": true
      }
    ],
    "uniqueDomains": [
      "docs.aws.amazon.com",
      "aws.amazon.com"
    ],
    "maxLinksReached": false
  }
}

Documentation

For more information, visit the API Documentation.

Support

License

This gem is available under the MIT License.