Class: Shugoi::BotVerifier

Inherits:
Object
  • Object
show all
Defined in:
lib/shugoi/bot_verifier.rb

Overview

Vérifie par DNS inverse + forward que les bots whitelistés viennent bien de leurs plages officielles — parité avec verify-bot.ts (module Node). Résultat mis en cache.

Constant Summary collapse

BOT_DOMAINS = 
[
  { pattern: /Googlebot|Google-InspectionTool|Storebot-Google/i, suffixes: [".googlebot.com", ".google.com"] },
  { pattern: /Bingbot|adidxbot|BingPreview/i, suffixes: [".search.msn.com"] },
  { pattern: /Slurp/i, suffixes: [".crawl.yahoo.net"] },
  { pattern: /DuckDuckBot/i, suffixes: [".duckduckgo.com"] },
  { pattern: /YandexBot/i, suffixes: [".yandex.ru", ".yandex.net", ".yandex.com"] },
  { pattern: /Applebot/i, suffixes: [".applebot.apple.com"] }
].freeze
VERIFY_TTL_MS = 
3_600_000
MAX_ENTRIES = 
5000

Instance Method Summary collapse

Constructor Details

#initialize ⇒ BotVerifier

Returns a new instance of BotVerifier.



21
22
23
24
 
# File 'lib/shugoi/bot_verifier.rb', line 21

def initialize
  @cache = {}
  @mutex = Mutex.new
end

Instance Method Details

#verify(ua, ip) ⇒ true, ...

Returns true/false si UA reconnu, nil sinon.

Parameters:

  • ua (String) —

    User-Agent

  • ip (String) —

    IP du client

Returns:

  • (true, false, nil) —

    true/false si UA reconnu, nil sinon



29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
 
# File 'lib/shugoi/bot_verifier.rb', line 29

def verify(ua, ip)
  entry = BOT_DOMAINS.find { |b| b[:pattern].match?(ua) }
  return nil unless entry
  return false if ip.to_s.empty? || ip == "unknown"

  key = "#{ip}|#{entry[:suffixes][0]}"
  @mutex.synchronize do
    hit = @cache[key]
    return hit[:ok] if hit && Utils.now_ms - hit[:at] < VERIFY_TTL_MS
  end

  ok = reverse_forward_match?(ip, entry[:suffixes])

  @mutex.synchronize do
    prune
    @cache[key] = { ok: ok, at: Utils.now_ms }
  end
  ok
end