Class: Protege::WebFetchTool

Inherits:

Tool

• Object
• Tool
• Protege::WebFetchTool

Defined in:

app/tools/protege/web_fetch_tool.rb

Overview

Built-in web-fetch tool — the way an agent reads a page or file from the open web. Subclasses Protege::Tool, so the harness publishes it in the LLM's tool catalog (id :web_fetch); when the model emits a web_fetch call, the harness routes the arguments to #use.

It is deliberately conservative, because the URL is agent-supplied and could be steered by prompt injection: only +http+/+https+ is allowed; hosts that resolve to private, loopback, or link-local addresses are refused (an SSRF guard, re-checked on every redirect) so the agent can never reach internal network endpoints; redirects are followed manually up to MAX_REDIRECTS; and the response is size-capped. HTML is reduced to readable text (tags stripped); other text-like content is returned as-is; binary content (images, PDFs, video) is refused — use read_attachment for files the agent already has.

Constant Summary

MAX_REDIRECTS =

Maximum number of redirect hops followed before giving up.

3

MAX_BYTES =

Hard ceiling (from the Content-Length header) above which the fetch is refused outright.

5_000_000

MAX_CONTENT_CHARS =

Returned text is truncated to this many characters (with truncated: true) so a huge page can never blow up the model's context.

100_000

OPEN_TIMEOUT =

Connection- and read-timeouts (seconds) for the HTTP request.

5

READ_TIMEOUT =

10

REDIRECT_STATUSES =

HTTP statuses treated as redirects (the Location header is followed).

[301, 302, 303, 307, 308].freeze

Instance Method Summary

• #use(context:, url:) ⇒ Protege::Result

  Fetch the URL and return its readable text.

Instance Method Details

#use(context:, url:) ⇒ Protege::Result

Fetch the URL and return its readable text.

Parameters:

• context (Protege::Orchestrator::Context) —

  the per-delivery tool context (unused)

• url (String) —

  the absolute http(s) URL supplied by the model

Returns:

• (Protege::Result) —

  success carrying the final url, status, content_type, content, and truncated flag, or a failure (bad scheme, blocked address, non-2xx, non-text, timeout, or too many redirects)

# File 'app/tools/protege/web_fetch_tool.rb', line 65

def use(context:, url:)
  fetch(url:, redirects_left: MAX_REDIRECTS)
rescue Faraday::Error => e
  failure(reason: "could not fetch #{url}: #{e.message}")
end