Module: CLI

Defined in:
lib/CLI.rb

Overview

All CLI-side concerns for the ‘ZMediumToMarkdown` executable. Pulled out of bin/ so it can be exercised by unit tests without spawning processes.

Constant Summary collapse 

'https://github.com/ZhgChgLi/ZMediumToMarkdown/wiki/Setting-Up-Medium-Cookies-and-a-Cloudflare-Worker-Proxy'.freeze
DEFAULT_MEDIUM_HOST = 
'https://medium.com/_/graphql'.freeze
DEFAULT_MIRO_MEDIUM_HOST = 
'https://miro.medium.com'.freeze

Class Method Summary collapse

Class Method Details

.buildSetupBanner(missingCookies:, missingProxy:, missingImageProxy:) ⇒ Object

Builds the dynamic setup-warning banner. Header lists exactly which of (cookies, GraphQL proxy, image proxy) is missing so the user can act; body is static guidance covering empirical limits, scenarios, and how to pass each value via flag or env.



157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
 
# File 'lib/CLI.rb', line 157

def buildSetupBanner(missingCookies:, missingProxy:, missingImageProxy:)
    lines = []
    lines << '──────────────────────────────────────────────────────────────────────'
    lines << '⚠  Setup notice — your run will work, but reliability is limited.'
    lines << ''
    lines << "What's missing:"
    lines << '  • Medium login cookies (sid / uid).' if missingCookies
    lines << '  • Cloudflare Worker proxy for Medium GraphQL (MEDIUM_HOST not set or still default).' if missingProxy
    lines << '  • Cloudflare Worker proxy for image CDN (MIRO_MEDIUM_HOST not set or still default; optional companion).' if missingImageProxy
    lines << ''
    lines << <<~BODY.chomp
      Empirical limits without setup:
        • Without cookies         : Cloudflare blocks after ~10 posts.
        • Without Worker proxy    : Cloudflare blocks after ~25 posts
                                    when running from CI / datacenter IPs.
        • Paywalled posts         : cookies are REQUIRED for full content;
                                    without them you only get the preview.

      Recommended setup:
        • CI / CD (GitHub Actions, cloud runners):
            STRONGLY recommend BOTH cookies AND a Cloudflare Worker proxy.
        • Local machine:
            Cookies recommended for paywalled posts. If a Cloudflare
            challenge appears, the tool will automatically open
            https://medium.com in your browser and prompt you to retry
            once you've cleared it. Set MEDIUM_NO_AUTO_BROWSER=1 to
            opt out and just fail fast.

      Pass cookies via env (preferred — keeps secrets out of shell history):
        MEDIUM_COOKIE_SID=... MEDIUM_COOKIE_UID=... ZMediumToMarkdown -p URL

      Or via flags (fine for one-off local runs):
        ZMediumToMarkdown -p URL -s YOUR_SID -d YOUR_UID

      Pass Cloudflare Worker proxy URL(s):
        ZMediumToMarkdown -p URL \\
          -x https://YOUR-WORKER.workers.dev/_/graphql \\
          --miro_medium_host https://YOUR-IMAGE-WORKER.workers.dev
        # or via env:
        #   MEDIUM_HOST=https://YOUR-WORKER.workers.dev/_/graphql
        #   MIRO_MEDIUM_HOST=https://YOUR-IMAGE-WORKER.workers.dev

      Full setup guide (cookies + Cloudflare Worker proxy):
        #{COOKIE_SETUP_URL}
      ──────────────────────────────────────────────────────────────────────
    BODY
    lines.join("\n")
end

.cookieMissing?(name) ⇒ Boolean

Returns:

  • (Boolean) 


112
113
114
115
 
# File 'lib/CLI.rb', line 112

def cookieMissing?(name)
    return true unless defined?($cookies) && $cookies.is_a?(Hash)
    $cookies[name].to_s.empty?
end

.cookiesPresent? ⇒ Boolean

Returns:

  • (Boolean) 


117
118
119
 
# File 'lib/CLI.rb', line 117

def cookiesPresent?
    !cookieMissing?('sid') || !cookieMissing?('uid')
end

.imageProxyConfigured? ⇒ Boolean

Returns:

  • (Boolean) 


129
130
131
132
 
# File 'lib/CLI.rb', line 129

def imageProxyConfigured?
    host = ENV['MIRO_MEDIUM_HOST'].to_s
    !host.empty? && host != DEFAULT_MIRO_MEDIUM_HOST
end

.loadCookiesFromEnv! ⇒ Object 



107
108
109
110
 
# File 'lib/CLI.rb', line 107

def loadCookiesFromEnv!
    $cookies['sid'] = ENV['MEDIUM_COOKIE_SID'] if cookieMissing?('sid') && !ENV['MEDIUM_COOKIE_SID'].to_s.empty?
    $cookies['uid'] = ENV['MEDIUM_COOKIE_UID'] if cookieMissing?('uid') && !ENV['MEDIUM_COOKIE_UID'].to_s.empty?
end

.main(argv, output: $stdout, errput: $stderr, cwd: ENV['PWD'] || ::Dir.pwd) ⇒ Object 



19
20
21
22
23
24
25
26
27
 
# File 'lib/CLI.rb', line 19

def main(argv, output: $stdout, errput: $stderr, cwd: ENV['PWD'] || ::Dir.pwd)
    argv = argv.dup
    argv << '-h' if argv.empty?

    options = parseArgs(argv, errput: errput)
    loadCookiesFromEnv!
    warnAboutMissingSetup(options, errput: errput)
    run(options, cwd, output: output, errput: errput)
end

.parseArgs(argv, errput: $stderr) ⇒ Object 



29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/CLI.rb', line 29

def parseArgs(argv, errput: $stderr)
    options = {}
    parser = OptionParser.new do |opts|
        opts.banner = "Usage: ZMediumToMarkdown [options]"

        opts.on('-s', '--cookie_sid SID', 'Medium logged-in cookie sid value (or set $MEDIUM_COOKIE_SID)') do |v|
            $cookies['sid'] = v
        end

        opts.on('-d', '--cookie_uid UID', 'Medium logged-in cookie uid value (or set $MEDIUM_COOKIE_UID)') do |v|
            $cookies['uid'] = v
        end

        opts.on('-x', '--medium_host URL', 'Cloudflare Worker proxy URL for Medium GraphQL (or set $MEDIUM_HOST). Strongly recommended for CI / bulk runs — see the wiki setup guide.') do |v|
            ENV['MEDIUM_HOST'] = v
        end

        opts.on('--miro_medium_host URL', 'Cloudflare Worker proxy URL for Medium image CDN (or set $MIRO_MEDIUM_HOST). Optional companion to --medium_host.') do |v|
            ENV['MIRO_MEDIUM_HOST'] = v
        end

        opts.on('-u', '--username USERNAME', 'Download all posts from a Medium username') do |v|
            options[:username] = v
        end

        opts.on('-p', '--postURL POST_URL', 'Download a single post URL') do |v|
            options[:postURL] = v
        end

        opts.on('--jekyll', 'Emit Jekyll-friendly output (combine with -u or -p)') do
            options[:jekyll] = true
        end

        opts.on('-j', '--jekyllUsername USERNAME', 'DEPRECATED: use `--jekyll -u USERNAME`') do |v|
            options[:username] = v
            options[:jekyll] = true
            errput.puts '[deprecated] -j/--jekyllUsername is deprecated; use `--jekyll -u USERNAME`.'
        end

        opts.on('-k', '--jekyllPostURL POST_URL', 'DEPRECATED: use `--jekyll -p POST_URL`') do |v|
            options[:postURL] = v
            options[:jekyll] = true
            errput.puts '[deprecated] -k/--jekyllPostURL is deprecated; use `--jekyll -p POST_URL`.'
        end

        opts.on('--stdout', 'Render Markdown of -p/-u directly to stdout. Skips all image/asset downloads (image links stay as remote URLs). Logs and banners go to stderr so stdout stays pure markdown.') do
            options[:stdout] = true
        end

        opts.on('--list', 'With -u <username>, emit one NDJSON line per post (title, url, creator, dates, tags) to stdout. Skips bodies and image downloads.') do
            options[:list] = true
        end

        opts.on('--limit N', Integer, 'Cap the number of posts processed when used with -u (in --stdout or --list mode).') do |v|
            options[:limit] = v
        end

        opts.on('-n', '--new', 'Update to latest version') do
            options[:upgrade] = true
        end

        opts.on('-c', '--clean', 'Remove all downloaded posts data under cwd') do
            options[:clean] = true
        end

        opts.on('-v', '--version', 'Print current ZMediumToMarkdown version') do
            options[:version] = true
        end

        opts.on('-h', '--help', 'Show this help message') do
            options[:help] = opts.to_s
        end
    end

    parser.parse!(argv)
    options
end

.pathPolicyFor(cwd, isForJekyll) ⇒ Object

Jekyll mode writes into the cwd (so files land in ‘_posts/…` and `assets/…` of an existing Jekyll site). Plain mode nests under `Output/` to keep the user’s cwd tidy.



284
285
286
287
288
289
290
 
# File 'lib/CLI.rb', line 284

def pathPolicyFor(cwd, isForJekyll)
    if isForJekyll
        PathPolicy.new(cwd, "")
    else
        PathPolicy.new("#{cwd}/Output", "Output")
    end
end

.proxyConfigured? ⇒ Boolean

Worker proxy is “configured” when MEDIUM_HOST is set to something other than the default upstream Medium URL — i.e. user pointed it at their own Cloudflare Worker (or another proxy).

Returns:

  • (Boolean) 


124
125
126
127
 
# File 'lib/CLI.rb', line 124

def proxyConfigured?
    host = ENV['MEDIUM_HOST'].to_s
    !host.empty? && host != DEFAULT_MEDIUM_HOST
end

.run(options, cwd, output: $stdout, errput: $stderr) ⇒ Object 



206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
 
# File 'lib/CLI.rb', line 206

def run(options, cwd, output: $stdout, errput: $stderr)
    if options[:help]
        output.puts options[:help]
        return
    end

    if options[:version]
        output.puts "Version:#{Helper.getLocalVersion()}"
        Helper.printNewVersionMessageIfExists()
        return
    end

    if options[:clean]
        outputFilePath = PathPolicy.new(cwd, "")
        FileUtils.rm_rf(Dir[outputFilePath.getAbsolutePath(nil)])
        output.puts "All downloaded posts data has been removed."
        Helper.printNewVersionMessageIfExists()
        return
    end

    if options[:upgrade]
        remote = Helper.getRemoteVersionFromGithub()
        local  = Helper.getLocalVersion()
        if remote && local && remote > local
            Helper.downloadLatestVersion()
        else
            output.puts "You're using the latest version :)"
        end
        return
    end

    # --stdout / --list path: render to the given output stream, skip
    # all filesystem writes and asset downloads. Progress goes to errput
    # so stdout stays pure markdown / NDJSON for embedding callers.
    # Handled before willHitMedium? so the --list-without-username guard
    # surfaces an error instead of silently no-op'ing.
    if options[:stdout] || options[:list]
        if options[:list] && options[:username].nil?
            errput.puts '--list requires -u/--username'
            return
        end
        return unless willHitMedium?(options)

        fetcher = ZMediumFetcher.new
        fetcher.isForJekyll = options[:jekyll] == true
        fetcher.stdoutIO = output
        fetcher.stdoutMode = true
        fetcher.progress.io = errput

        if options[:list]
            fetcher.listPostsByUsername(options[:username], options[:limit])
        elsif options[:postURL]
            fetcher.downloadPost(options[:postURL], nil, nil)
        elsif options[:username]
            fetcher.downloadPostsByUsername(options[:username], nil, limit: options[:limit])
        end
        return
    end

    return unless willHitMedium?(options)

    fetcher = ZMediumFetcher.new
    fetcher.isForJekyll = options[:jekyll] == true

    targetPolicy = pathPolicyFor(cwd, fetcher.isForJekyll)

    if options[:postURL]
        fetcher.downloadPost(options[:postURL], targetPolicy, nil)
    elsif options[:username]
        fetcher.downloadPostsByUsername(options[:username], targetPolicy, limit: options[:limit])
    end

    Helper.printNewVersionMessageIfExists()
end

.warnAboutMissingSetup(options, errput: $stderr) ⇒ Object

Only warn when the invocation will actually hit Medium — skip for –version, –clean, –help, –new.



136
137
138
139
140
141
142
143
144
145
146
147
 
# File 'lib/CLI.rb', line 136

def warnAboutMissingSetup(options, errput: $stderr)
    return unless willHitMedium?(options)

    missingCookies     = !cookiesPresent?
    missingProxy       = !proxyConfigured?
    missingImageProxy  = !imageProxyConfigured?
    return if !missingCookies && !missingProxy && !missingImageProxy

    errput.puts buildSetupBanner(missingCookies: missingCookies,
                                 missingProxy: missingProxy,
                                 missingImageProxy: missingImageProxy)
end

.willHitMedium?(options) ⇒ Boolean

Returns:

  • (Boolean) 


149
150
151
 
# File 'lib/CLI.rb', line 149

def willHitMedium?(options)
    !options[:postURL].nil? || !options[:username].nil?
end