Class: Abqari::Importers::Substack

Inherits:
Base
  • Object
show all
Defined in:
lib/abqari/importers/substack.rb

Overview

Substack export importer.

Substack's export shape (as of 2025–2026):

substack-export/
posts.csv              # post_id, post_date, title, subtitle,
                       #   type, audience, status, slug,
                       #   email_sent_at, ...
posts/
  12345.html
  67890.html

Some columns vary across export vintages (Substack rolls schema changes silently). The reader is tolerant — we look up columns by name, default missing ones to nil, and never assume column ordering.

Source accepts both a .zip (auto-extracted to a tmp dir) and an already-unzipped directory. Zip extraction uses the shell unzip so we don't pull in rubyzip as a dep — the operator is running this from a CLI environment that has unzip on every Mac/Linux system.

Instance Attribute Summary

Attributes inherited from Base

#report

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

#initialize, #run!, short_name

Constructor Details

This class inherits a constructor from Abqari::Importers::Base

Class Method Details

.strip_selectors ⇒ Object

Substack chrome to strip before HTML→Markdown conversion. Each entry is a tag.class pair that HtmlToMarkdown can parse — see Support::HtmlToMarkdown.parse_selector.

These selectors come from inspecting the actual HTML Substack ships in 2025 exports. New chrome blocks appear periodically; users can add their own via --strip-selectors without forking this list.



41
42
43
44
45
46
47
48
49
50
51
52
53
 
# File 'lib/abqari/importers/substack.rb', line 41

def self.strip_selectors
  %w[
    div.subscription-widget-wrap
    div.subscription-widget
    div.button-wrapper
    div.footnote-hovercard
    aside.poll
    aside.subscribe-widget
    div.embedded-publication-wrap
    div.share-dialog
    div.captioned-button-wrap
  ]
end

Instance Method Details

#each_post ⇒ Object 



55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
 
# File 'lib/abqari/importers/substack.rb', line 55

def each_post
  with_export_root do |root|
    manifest_path = File.join(root, 'posts.csv')
    unless File.exist?(manifest_path)
      raise "Substack export missing posts.csv at #{manifest_path}"
    end

    CSV.foreach(manifest_path, headers: true, encoding: 'UTF-8') do |row|
      next unless publishable?(row)

      html_path = File.join(root, 'posts', "#{row['post_id']}.html")
      unless File.exist?(html_path)
        # Some rows in posts.csv have no corresponding HTML
        # file (email-only sends, draft cleanup). Skip rather
        # than fail the run.
        next
      end

      body_html = File.read(html_path, encoding: 'UTF-8')
      yield Base::ImportedPost.new(
        title:             row['title'],
        slug:              row['slug'] || derive_slug_from_title(row['title']),
        date:              parse_date(row['post_date']),
        body_html:         body_html,
        tags:              [], # Substack export doesn't include tags in posts.csv
        description:       row['subtitle'],
        feature_image_url: nil, # Substack inlines hero images in the body
        source_url:        row['post_url'],
        source_slug:       row['slug'],
        extra_frontmatter: substack_extras(row)
      )
    end
  end
end