Class: RailsErrorDashboard::Services::AiAgentClassifier

Inherits:
Object
  • Object
show all
Defined in:
lib/rails_error_dashboard/services/ai_agent_classifier.rb

Overview

Classifies a User-Agent string into a coarse traffic kind, and names the agent when it is a recognised one.

WHY THIS EXISTS (issue #170): tracking which AI agents read an app is the reason people reach for Rack::Attack's track rules now. Counting IPs cannot answer it — one agent is a rotating fleet of hundreds of addresses, so unique-IP totals overstate the population badly. The user agent is the signal that actually identifies the reader.

Deliberately plain string matching, NOT the browser gem: browser is an optional dependency that degrades gracefully everywhere else in this gem, and it does not know these agents anyway. This runs on the flush path, not the request path, but it stays allocation-cheap regardless.

The bot lists are necessarily a snapshot. An unrecognised agent falls back to :other rather than being guessed at — a wrong attribution is worse than an honest "unknown" when the whole point is measurement.

Constant Summary collapse

AI_ASSISTANTS =

Order matters: the first match wins, so more specific patterns lead.

AI agents split into two behaviours worth telling apart, because they answer different questions:

  • :ai_assistant — fetches on demand, because a human asked something now
  • :ai_crawler — bulk-fetches to build a training corpus or index
{
  "ChatGPT-User" => /ChatGPT-User/i,
  "Claude-User" => /Claude-User/i,
  "Claude Code" => /Claude-?Code/i,
  "Perplexity-User" => /Perplexity-User/i,
  "Gemini-User" => /Gemini-User/i,
  # Observed in production traffic by the reporter of #170:
  # "GitHubCopilotRuntime-WebFetch", 7 requests from 5 IPs. Classified as
  # an assistant rather than a crawler because it fetches a specific URL a
  # developer's Copilot session asked for, one page at a time — not a bulk
  # corpus crawl.
  #
  # The pattern is deliberately anchored on "GitHubCopilot" rather than a
  # bare /Copilot/i. Microsoft applies the Copilot brand very broadly, and
  # its Copilot features are documented as riding Bingbot infrastructure or
  # sending ordinary Edge/Chromium user agents with no bot signal — so a
  # loose pattern would mislabel plain browser traffic as an AI agent,
  # which is exactly the wrong-attribution failure this class avoids.
  #
  # No authoritative UA documentation exists for this agent: it is absent
  # from GitHub's own docs, from ai-robots-txt/ai.robots.txt, and from the
  # Dark Visitors catalogue as of 2026-08-29. The pattern therefore matches
  # only what has actually been observed, plus the "-WebFetch" sibling
  # suffix, instead of guessing at variants.
  "GitHub Copilot" => /GitHubCopilot/i
}.freeze
AI_CRAWLERS = 
{
  "GPTBot" => /GPTBot/i,
  "OAI-SearchBot" => /OAI-SearchBot/i,
  "ClaudeBot" => /ClaudeBot/i,
  "anthropic-ai" => /anthropic-ai/i,
  "PerplexityBot" => /PerplexityBot/i,
  "Google-Extended" => /Google-Extended/i,
  "Applebot-Extended" => /Applebot-Extended/i,
  "Bytespider" => /Bytespider/i,
  "CCBot" => /CCBot/i,
  "Meta-ExternalAgent" => /Meta-ExternalAgent/i,
  "Amazonbot" => /Amazonbot/i,
  "cohere-ai" => /cohere-ai/i,
  "DuckAssistBot" => /DuckAssistBot/i,
  "YouBot" => /YouBot/i,
  "Diffbot" => /Diffbot/i,
  "Timpibot" => /Timpibot/i
}.freeze
CRAWLERS =

Conventional search/SEO crawlers. Not AI traffic, but worth naming so they can be excluded rather than silently inflating an "unknown" bucket.

{
  "Googlebot" => /Googlebot/i,
  "Bingbot" => /bingbot/i,
  "DuckDuckBot" => /DuckDuckBot/i,
  "Baiduspider" => /Baiduspider/i,
  "YandexBot" => /YandexBot/i,
  "AhrefsBot" => /AhrefsBot/i,
  "SemrushBot" => /SemrushBot/i,
  "Applebot" => /Applebot/i,
  "facebookexternalhit" => /facebookexternalhit/i,
  "LLMS-Txt-Scanner" => /LLMS-Txt-Scanner/i
}.freeze
BROWSER_HINTS =

Checked only after every bot pattern has missed, because plenty of bots embed a full browser UA string and would match these first.

/Mozilla|Chrome|Safari|Firefox|Edge|Opera|Gecko|WebKit/i
LIBRARIES =

Non-browser HTTP clients — usually scripts, monitors or scrapers.

{
  "curl" => /\bcurl\//i,
  "wget" => /\bWget\//i,
  "python-requests" => /python-requests/i,
  "httpx" => /\bhttpx\//i,
  "Go-http-client" => /Go-http-client/i,
  "Java" => /\bJava\//i,
  "okhttp" => /\bokhttp\//i,
  "axios" => /\baxios\//i,
  "Faraday" => /Faraday/i,
  "RubyGems" => /Ruby\b/i
}.freeze
KINDS = 
%i[ai_assistant ai_crawler crawler browser library other].freeze

Class Method Summary collapse

Class Method Details

.ai?(user_agent) ⇒ Boolean

Whether this agent is an LLM reader of either flavour. This is the predicate the dashboard's "AI agents" figure counts.

Parameters:

  • user_agent (String, nil)

Returns:

  • (Boolean) 


150
151
152
 
# File 'lib/rails_error_dashboard/services/ai_agent_classifier.rb', line 150

def ai?(user_agent)
  %i[ai_assistant ai_crawler].include?(kind(user_agent))
end

.classify(user_agent) ⇒ Hash

Returns { kind:, name:, ai: } — one pass for callers that want all three.

Returns:

  • (Hash) —

    { kind:, name:, ai: } — one pass for callers that want all three



155
156
157
158
159
160
161
162
 
# File 'lib/rails_error_dashboard/services/ai_agent_classifier.rb', line 155

def classify(user_agent)
  k = kind(user_agent)
  {
    kind: k,
    name: name(user_agent),
    ai: %i[ai_assistant ai_crawler].include?(k)
  }
end

.kind(user_agent) ⇒ Symbol

Returns one of KINDS.

Parameters:

  • user_agent (String, nil)

Returns:

  • (Symbol) —

    one of KINDS



113
114
115
116
117
118
119
120
121
122
123
124
125
126
 
# File 'lib/rails_error_dashboard/services/ai_agent_classifier.rb', line 113

def kind(user_agent)
  ua = user_agent.to_s
  return :other if ua.strip.empty?

  return :ai_assistant if match_name(AI_ASSISTANTS, ua)
  return :ai_crawler if match_name(AI_CRAWLERS, ua)
  return :crawler if match_name(CRAWLERS, ua)
  return :library if match_name(LIBRARIES, ua)
  return :browser if ua.match?(BROWSER_HINTS)

  :other
rescue => e
  :other
end

.name(user_agent) ⇒ String?

Canonical name for a recognised agent, or nil when unrecognised. Never invents a name — callers show the raw UA in that case.

Parameters:

  • user_agent (String, nil)

Returns:

  • (String, nil) 


133
134
135
136
137
138
139
140
141
142
143
 
# File 'lib/rails_error_dashboard/services/ai_agent_classifier.rb', line 133

def name(user_agent)
  ua = user_agent.to_s
  return nil if ua.strip.empty?

  match_name(AI_ASSISTANTS, ua) ||
    match_name(AI_CRAWLERS, ua) ||
    match_name(CRAWLERS, ua) ||
    match_name(LIBRARIES, ua)
rescue => e
  nil
end