Class: Abqari::Renderer

Inherits:
Object
  • Object
show all
Defined in:
lib/abqari/renderer.rb

Overview

Page-rendering collaborator. Owns:

- Generators that produce virtual pages BEFORE render
(tag pages, paginated shards, visualizations, Folio pages,
related-index pages).
- The render loop itself, with parallel execution and per-page
error collection.
- ERB template caching (shared across threads via a mutex).
- The shared-deps mtime used by incremental rendering.
- HTML post-processing (bundle-URL rewrite, syntax highlighting,
responsive image rewrites, minification, livereload injection).

Constant Summary collapse

PARALLEL_THRESHOLD =

Below this many pages, parallelism hurts more than it helps.

16
PAGINATION_DEFAULT =

Default page-size for paginate: true. Overridable per-site via pagination.per_page config.

10
SLOW_PAGE_THRESHOLD_MS =

Slow-page detection. When a single page's write (render + post-process + file write) takes longer than this, the renderer logs a warning with the URL, elapsed time, and a short list of heuristic causes (output size, image count, code-block count, TOC presence). End-of-build summary surfaces the slowest few.

Default 500ms — high enough to ignore the normal per-page baseline (~5–50ms), low enough to catch real outliers. Override with ABQARI_SLOW_PAGE_MS=N (set to 0 to disable detection).

(ENV['ABQARI_SLOW_PAGE_MS'] || '500').to_i
GLOBAL_DEP_GLOBS =

Files whose mtime invalidates every cached page render. app/views/layouts/ is NOT here — layout files are handled per-layout via layout_deps_mtime(name) so editing one layout only invalidates the pages that use it.

Partials stay in the global bucket: statically tracking which partials each layout chain renders would require scanning ERB source for render '...' calls and is more complexity than the win is worth at the scale Abqari targets. Edits to a partial therefore still invalidate every cached render.

themes/**/* is NOT in this list — it's resolved per-site at compute_shared_deps_mtime time so we only walk the active theme's directory, not every theme that happens to ship with the gem.

[
  'app/views/partials/**/*',
  'app/icons/**/*',
  'app/assets/**/*',
  'lib/**/*',
  'config/**/*',
  'data/**/*'
].freeze
LAYOUT_FRONTMATTER_RE =

The renderer only needs the frontmatter block (no body), but reusing Abqari::FRONTMATTER_RE and discarding match[2] keeps the three call sites in lockstep — the old duplicate-with-drift was finding a --- pair that the page parser wouldn't.

Abqari::FRONTMATTER_RE

Instance Method Summary collapse

Constructor Details

#initialize(site) ⇒ Renderer

Returns a new instance of Renderer.



66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/abqari/renderer.rb', line 66

def initialize(site)
  @site = site
  @template_cache = {}
  @template_mutex = Mutex.new
  # Separate mutex for the mtime / layout-chain caches. Different
  # contention profile from the template cache (computed once per
  # layout name, then read-only for the rest of the build), and
  # keeping the mutex separate means a slow first-miss on the
  # mtime side doesn't block template compilation.
  @deps_mutex     = Mutex.new
  @layout_chains  = {}
  @layout_mtimes  = {}
end

Instance Method Details

#clear_template_cache ⇒ Object

Reset between builds (incremental dev server keeps the Site around across rebuilds; the template cache must clear so layout edits take effect). Layout-chain caches are reset too — a new layout file or a frontmatter layout: edit changes the chain.



84
85
86
87
88
89
90
91
 
# File 'lib/abqari/renderer.rb', line 84

def clear_template_cache
  @template_mutex.synchronize { @template_cache = {} }
  @deps_mutex.synchronize do
    @shared_deps_mtime = nil
    @layout_chains     = {}
    @layout_mtimes     = {}
  end
end

#compiled_template(path) ⇒ Object

Lazily compile and cache a template. Safe to call from threads. Uses Erubi with escape: true so every <%= %> interpolation is auto-escaped; partials emit raw markup via <%== %>. See ErbiTemplate for the rationale.



97
98
99
100
101
 
# File 'lib/abqari/renderer.rb', line 97

def compiled_template(path)
  @template_mutex.synchronize do
    @template_cache[path] ||= ErbiTemplate.new(File.read(path, encoding: 'UTF-8'), path)
  end
end

#generate_all ⇒ Object

Two phases, and the order between them is load-bearing.

CONTRIBUTE — generators that add pages which JOIN a collection. PublisherShowPage is the case that exists today: Folio publications, bundles and series expose #collection, so Site#collection merges them with the manual content pages.

DERIVE — generators that READ collections and freeze what they find. Pagination slices a collection into Paginator objects at generation time; whatever isn't in the collection when that runs is absent from the archive and every shard, permanently.

Pagination used to run before publisher pages, so on a Folio site with more manual publications than per_page, every Folio-sourced publication silently vanished from the archive — no warning, no error, just missing entries. Site#finalise_collections! makes a repeat of that a hard failure instead of a silent one: after the barrier, adding a page that joins a collection raises.

Tag pages sit in DERIVE for ordering hygiene, though they read @pages (source pages only, see Site#taxonomy_pages) and so were never affected by this.



161
162
163
164
165
166
167
168
169
170
 
# File 'lib/abqari/renderer.rb', line 161

def generate_all
  generate_publisher_pages

  @site.finalise_collections!

  generate_tag_pages
  generate_paginated_pages
  generate_visualizations
  generate_related_pages
end

#generate_paginated_pages ⇒ Object 



207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
 
# File 'lib/abqari/renderer.rb', line 207

def generate_paginated_pages
  @site.pages.each do |page|
    # Skip cheaply BEFORE resolving the collection — pages with no
    # `paginate:` key (or `paginate: false`) don't paginate and
    # don't need the collection lookup.
    raw = page.frontmatter['paginate']
    next if raw.nil? || raw == false

    # Resolve which collection this page paginates. Four signals,
    # first match wins:
    #   1. Pages inside a collection paginate that collection.
    #   2. The index page OF a collection paginates its own collection.
    #   3. Frontmatter `paginate_collection:`.
    #   4. Final fallback: posts.
    coll_name = page.collection ||
                @site.collection_index_name(page) ||
                page.frontmatter['paginate_collection'] ||
                'posts'

    # Now resolve per_page WITH the collection in scope so
    # `paginate: true` can pick up a per-collection default.
    per_page = resolve_per_page(page, coll_name)
    next unless per_page

    items = @site.collection(coll_name)

    slices = items.each_slice(per_page).to_a
    next if slices.size <= 1

    page.paginator = Paginator.new(
      posts: slices.first,
      page: 1,
      total_pages: slices.size,
      base_url: page.url
    )

    slices[1..].each_with_index do |slice, idx|
      shard_paginator = Paginator.new(
        posts: slice,
        page: idx + 2,
        total_pages: slices.size,
        base_url: page.url
      )
      @site.add_generated_page(PaginatedShard.new(page, shard_paginator))
    end
  end
end

#generate_publisher_pages ⇒ Object

Auto-generate /books/, /bundles/, /series/ index + show pages from an external publisher's data (currently Folio — see folio: config). No-op when no publisher is configured. Pages already defined via content/ markdown files take precedence.



188
189
190
 
# File 'lib/abqari/renderer.rb', line 188

def generate_publisher_pages
  PublisherPages.new(@site).pages.each { |page| @site.add_generated_page(page) }
end

#generate_related_pages ⇒ Object

Generate /<source url>related/ index pages for any show-view page whose related list is longer than its visible limit.



194
195
196
197
198
199
200
201
202
203
204
205
 
# File 'lib/abqari/renderer.rb', line 194

def generate_related_pages
  @site.pages.select { |p| p.respond_to?(:collection) && p.collection }.each do |page|
    cfg = @site.collection_config(page.collection).dig('show', 'related') || {}
    next if cfg['enabled'] == false

    all = @site.related(page)
    visible_limit = cfg.fetch('limit', 3)
    next if all.size <= visible_limit

    @site.add_generated_page(RelatedIndexPage.new(page, @site, all))
  end
end

#generate_tag_pages ⇒ Object 



172
173
174
175
176
177
178
 
# File 'lib/abqari/renderer.rb', line 172

def generate_tag_pages
  @site.taxonomies.each_key do |name|
    @site.posts_by_term(name).each do |term, term_posts|
      @site.add_generated_page(TaxonomyPage.new(name, term, term_posts, @site))
    end
  end
end

#generate_visualizations ⇒ Object 



180
181
182
 
# File 'lib/abqari/renderer.rb', line 180

def generate_visualizations
  @site.visualizations.pages.each { |page| @site.add_generated_page(page) }
end

#layout_deps_mtime(layout_name) ⇒ Object

Most-recent mtime across layout_name's file plus every ancestor in its layout: chain. Memoised per layout name. Page#needs_render? combines this with shared_deps_mtime so editing layouts/post.html.erb only invalidates pages that resolve to the post layout (or something that extends it), not photo / publication / workshop pages.

Mutex-protected for the same reason as shared_deps_mtime: called from the parallel render loop, and a torn write of the underlying hash would be observable on Ruby implementations without GIL serialisation (TruffleRuby, JRuby).



128
129
130
131
132
133
 
# File 'lib/abqari/renderer.rb', line 128

def layout_deps_mtime(layout_name)
  key = layout_name.to_s
  @deps_mutex.synchronize do
    @layout_mtimes[key] ||= compute_layout_deps_mtime(key)
  end
end

#minify_html? ⇒ Boolean

Minify rendered HTML in production by default; disable with minify: false in site.yml. Off in development so livereload + view-source stay readable.

Returns:

  • (Boolean) 


343
344
345
346
347
 
# File 'lib/abqari/renderer.rb', line 343

def minify_html?
  return @site.config['minify'] if @site.config.key?('minify')

  @site.env == :production
end

#post_process(html, page: nil) ⇒ Object

Post-process



323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
 
# File 'lib/abqari/renderer.rb', line 323

def post_process(html, page: nil)
  # `rewrite_bundle_urls` runs before `images.post_process_html` so
  # the latter sees absolute paths (`/posts/foo/hero.jpg`) and can
  # match them against the URL → source registry.
  html = rewrite_bundle_urls(html, page) if page&.respond_to?(:post?) && page.post?
  html = SyntaxHighlighter.process(html) if @site.config['syntax_highlighting']
  html = @site.images.post_process_html(html, page: page) if @site.images.enabled?
  # Plugin filter — runs after the engine's built-in post-processing
  # so plugin output sees the same shape browsers will. Runs BEFORE
  # livereload injection + minification so plugin transformations
  # are minified along with the rest of the page.
  html = Hooks.run_filter(:post_process, html, @site, page)
  html = inject_livereload(html) if @site.env == :development
  html = Minifier.minify(html) if minify_html?
  html
end

#render_pages ⇒ Object

Render pages in parallel across CPU cores. ERB rendering does enough I/O (template reads, asset_path lookups, write to disk) that threading helps despite the GIL. For tiny sites the thread-spawn overhead exceeds the benefit, so fall through to a serial loop below the threshold.

In incremental mode, pages whose output is up to date are skipped via Page#needs_render?.

Scheduling: a Thread::Queue work-stealing pool. Earlier slices- based partitioning evenly split the page list, but render cost varies wildly (a post with a TOC, bundle picture, syntax highlighting, and minified output costs far more than a flat tag page) so the slowest slice dominated total time. With a queue, whichever thread finishes its current page picks the next one up immediately.



275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
 
# File 'lib/abqari/renderer.rb', line 275

def render_pages
  detect_output_collisions!

  pages = @site.incremental? ? @site.all_pages.select(&:needs_render?) : @site.all_pages
  return if pages.empty?

  errors = []
  errors_mutex = Mutex.new
  slow_pages = []
  slow_mutex = Mutex.new

  if pages.size < PARALLEL_THRESHOLD
    pages.each { |page| render_one(page, errors, errors_mutex, slow_pages, slow_mutex) }
    report_slow_pages!(slow_pages)
    return report_render_errors!(errors)
  end

  queue = Thread::Queue.new
  pages.each { |page| queue << page }

  thread_count = [Etc.nprocessors, pages.size].min
  threads = Array.new(thread_count) do
    Thread.new do
      loop do
        # `pop(true)` is non-blocking — it raises ThreadError when the
        # queue is empty, which is the worker's signal to exit. This
        # is the work-stealing primitive: every worker keeps pulling
        # until there's nothing left, regardless of how many pages
        # other workers have already finished.
        page = begin
          queue.pop(true)
        rescue ThreadError
          break
        end
        render_one(page, errors, errors_mutex, slow_pages, slow_mutex)
      end
    end
  end
  threads.each(&:join)

  report_slow_pages!(slow_pages)
  report_render_errors!(errors)
end

#shared_deps_mtime ⇒ Object

Most-recent mtime across the global dep set (everything except layout files). When this is newer than a page's output, that output is stale regardless of which layout the page uses.

Mutex-protected. Page#needs_render? runs inside the parallel render loop, so concurrent threads can race on the first miss — without the mutex they'd each pay the (expensive) directory- walk cost. With it, one thread computes, the rest wait.



111
112
113
114
115
 
# File 'lib/abqari/renderer.rb', line 111

def shared_deps_mtime
  @deps_mutex.synchronize do
    @shared_deps_mtime ||= compute_shared_deps_mtime
  end
end