Class: Abqari::Importers::Support::ImageDownloader

Inherits:
Object
  • Object
show all
Defined in:
lib/abqari/importers/support/image_downloader.rb

Overview

Downloads remote images into a post's bundle directory and returns a map of {original_url => local_filename} so the caller can rewrite references in the post body.

Why download at all: source-platform CDN URLs (substackcdn.com, ghost.io's blob storage, Medium's image proxy) get rotated or revoked periodically. Sites that keep remote URLs look fine immediately and then break six to twenty-four months later when the source decides to expire the link. Downloading at import-time is slower but durable — the image is now part of the post's bundle and goes through Abqari's image pipeline at build time (AVIF + WebP variants, metadata stripped, responsive <picture> rewriting). The pipeline runs the same way it does for a hand-authored bundle, so an imported post ends up indistinguishable from one written natively.

Failure handling: a download failure is reported as a warning and the URL is left as-is in the body. We never abort the whole post because of an image — most posts have many images and one CDN hiccup shouldn't sink the whole import.

Constant Summary collapse

IMAGE_EXTENSIONS =

Extensions we recognise as images. Anything else is left in place (svg passes through here because Abqari's image pipeline preserves it; non-image extensions like .pdf are intentionally ignored so we don't accidentally try to "download an image" that's actually a document).

%w[.jpg .jpeg .png .gif .webp .avif .svg].freeze
MAX_IMAGE_BYTES =

20 MB cap per image. Larger than the engine's default 8 MB because some Substack hero images come through close to 10–15 MB pre-optimisation. The build's image pipeline will downsize them after import.

20 * 1024 * 1024

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(bundle_dir:, logger: nil) ⇒ ImageDownloader

Returns a new instance of ImageDownloader.



46
47
48
49
50
51
52
 
# File 'lib/abqari/importers/support/image_downloader.rb', line 46

def initialize(bundle_dir:, logger: nil)
  @bundle_dir = bundle_dir
  @logger     = logger
  @warnings   = []
  @cache      = {} # url → local filename, so repeated refs share a file
  @taken      = Set.new # local filenames already used in this bundle
end

Instance Attribute Details

#warnings ⇒ Object (readonly)

Returns the value of attribute warnings.



44
45
46
 
# File 'lib/abqari/importers/support/image_downloader.rb', line 44

def warnings
  @warnings
end

Instance Method Details

#fetch(url) ⇒ Object

Resolve a remote image URL to a local filename. On success, downloads the file into the bundle dir and returns the local filename. On failure, records a warning and returns nil (caller leaves the URL alone).



58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/abqari/importers/support/image_downloader.rb', line 58

def fetch(url)
  return nil if url.to_s.empty?
  return @cache[url] if @cache.key?(url)

  unless url.match?(%r{\Ahttps?://}i)
    # Not a remote URL — caller should leave it alone.
    return nil
  end

  local = download(url)
  @cache[url] = local
  local
rescue StandardError => e
  warn("Failed to download #{url}: #{e.class}: #{e.message}")
  @cache[url] = nil
  nil
end