Durable Streams Rails

Durable Streams integration for Rails. Stream State Protocol events from your models to clients over HTTP with the same developer experience as Turbo Broadcasts — but with offset-based resumability, persistent event logs, and no WebSocket infrastructure.

class Comment < ApplicationRecord
  belongs_to :post
  streams_to :post
end

That's it. Creates, updates, and destroys are broadcast as State Protocol events to all clients subscribed to the post's stream. No manual stream management, no JSON serialization, no offset tracking.

Installation

Add to your Gemfile:

gem "durable_streams-rails"

Run the install generator:

bin/rails generate durable_streams:install

This creates:

• config/initializers/durable_streams.rb — sets client_url and server_url
• config/durable_streams.yml — server configuration (per environment)
• bin/durable-streams — binstub that auto-downloads and runs the server binary
• Updates Procfile.dev with a streams: entry
• Updates .gitignore to exclude bin/dist/

Pin a server version

bin/rails generate durable_streams:install --version=0.1.0

Configuration

JWT signing

The gem uses asymmetric ES256 JWTs. Rails signs tokens with a private key, Caddy verifies with the public key via ggicci/caddy-jwt — no callback to Rails.

# config/initializers/durable_streams.rb
Rails.application.config.durable_streams.signing_key  = Rails.application.credentials.dig(:durable_streams, :signing_key)
Rails.application.config.durable_streams.signing_kid  = Rails.application.credentials.dig(:durable_streams, :signing_kid)
Rails.application.config.durable_streams.token_issuer = "https://your-app.com"

Generate a key pair:

bin/rails durable_streams:generate_keys

Server config

The server reads config/durable_streams.yml:

default: &default
  port: 4437
  auth:
    verify_key: config/caddy/verify_key.pem
    issuer: https://localhost:3000
    audience: https://localhost:4437/v1/streams

development:
  <<: *default

production:
  domain: streams.example.com
  tls:
    cert: /etc/caddy/certs/cert.pem
    key: /etc/caddy/certs/key.pem
  auth:
    verify_key: /etc/durable_streams/verify_key.pem
    issuer: https://your-app.com
    audience: https://streams.example.com/v1/streams
  storage:
    data_dir: /var/data/durable-streams

Multi-server with internal listener

In production, you typically want browsers to reach the stream server through a CDN (client_url), but Rails broadcasts should go directly over the private network (server_url). The internal option adds a plain HTTP listener for server-to-server traffic:

production:
  domain: streams.example.com
  tls:
    cert: /etc/caddy/certs/cert.pem
    key: /etc/caddy/certs/key.pem
  internal:
    port: 4437
    bind: 10.0.0.5
    allowed_ips:
      - 10.0.0.4
  auth:
    verify_key: /etc/durable_streams/verify_key.pem
    issuer: https://your-app.com
    audience: https://streams.example.com/v1/streams
  storage:
    data_dir: /data/streams

Request flow

A server-to-server write (e.g., Rails broadcasting a message) follows this path:

Rails (10.0.0.4)
  │
  │  PUT http://10.0.0.5:4437/v1/streams/{name}
  │  (plain HTTP over private VNET — no TLS, no CDN)
  │
  ▼
:4437 listener (Caddy on 10.0.0.5)
  │
  │  1. remote_ip check: is it 10.0.0.4? → proceed (otherwise 403)
  │
  │  2. reverse_proxy to https://127.0.0.1:443
  │     Caddy connects to itself over loopback TLS:
  │
  │     header_up Host streams.example.com
  │       → Host header so :443 matches the right server block
  │
  │     tls_server_name streams.example.com
  │       → SNI in the TLS ClientHello so Caddy selects the right cert
  │
  │     tls_insecure_skip_verify
  │       → Skip cert verification (origin certs aren't in the system
  │         trust store; safe on loopback — nothing to MITM)
  │
  ▼
streams.example.com :443 listener (same Caddy process)
  │
  │  jwtauth (verifies ES256 JWT locally using public key — no callback to Rails)
  │  durable_streams → writes to bbolt store
  │
  ▼
Response flows back up the chain to Rails

Client reads (browsers) go directly to streams.example.com:443 through the CDN, bypassing the internal listener entirely.

Why reverse_proxy instead of a second durable_streams?

The stream storage engine (bbolt) is single-writer — two handler instances can't open the same database file. The second one deadlocks waiting for the write lock. The internal listener avoids this by proxying to the main listener over loopback, so only one durable_streams handler exists process-wide.

Power users can bypass the YAML entirely:

DURABLE_STREAMS_CONFIG=/path/to/Caddyfile bin/durable-streams

Deployment with Kamal

The server binary runs as a separate role in the same Docker image — same pattern as Solid Queue:

# config/deploy.yml
servers:
  web:
    cmd: bin/rails server
  jobs:
    cmd: bin/jobs
  streams:
    cmd: bin/durable-streams

Pre-download the binary during Docker build:

RUN bin/rails durable_streams:download

Or set a specific version:

RUN DURABLE_STREAMS_VERSION=0.1.0 bin/rails durable_streams:download

Usage

Declarative streaming

Stream to an association:

class Comment < ApplicationRecord
  belongs_to :post
  streams_to :post
end

Stream to self:

class Board < ApplicationRecord
  streams
end

With a proc for custom stream targets:

class Comment < ApplicationRecord
  belongs_to :post
  streams_to ->(comment) { [comment.post, :comments] }
end

Instance methods

Synchronous (returns txid for optimistic update confirmation):

txid = comment.stream_insert_to post
txid = comment.stream_update_to post
txid = comment.stream_upsert_to post
comment.stream_delete_to post

Asynchronous (via DurableStreams::Rails::BroadcastJob):

comment.stream_insert_later_to post
comment.stream_update_later_to post
comment.stream_upsert_later_to post
comment.stream_delete_later_to post

Self-targeting (streams to the model itself):

comment.stream_insert          # same as stream_insert_to(self)
comment.stream_update_later    # same as stream_update_later_to(self)

Direct broadcasting

For non-ActiveRecord use cases (presence, typing indicators):

DurableStreams.broadcast_to(room, :presence,
  type: "presence",
  key: user.id.to_s,
  value: { id: user.id, name: user.name },
  headers: { operation: "insert" }
)

Signed stream URLs

Generate signed, expirable URLs for client streaming connections:

url = DurableStreams.signed_stream_url(room, :messages)
# => "http://localhost:4437/v1/streams/gid://app/Room/1/messages?token=eyJ..."

url = DurableStreams.signed_stream_url(room, :messages, expires_in: 1.hour)

Suppressing broadcasts

Comment.suppressing_streams do
  Comment.create!(post: post) # no broadcast
end

Testing

The gem provides test helpers that mirror Turbo's assert_turbo_stream_broadcasts. They are automatically included in ActiveSupport::TestCase via the engine.

During tests, DurableStreams::Rails::Testing intercepts all broadcasts at the transport layer — events are captured in-memory instead of being sent to the stream server. This means tests verify the Rails integration (DSL wiring, event shape, callbacks, jobs, suppression) without requiring a running Durable Streams server. Same pattern as Turbo, where ActionCable::TestHelper captures broadcasts without a WebSocket connection.

class CommentTest < ActiveSupport::TestCase
  test "creating comment streams to post" do
    assert_stream_broadcasts @post, count: 1 do
      @post.comments.create!(body: "Hello")
    end
  end

  test "capture stream events" do
    events = capture_stream_broadcasts @post do
      @post.comments.create!(body: "Hello")
    end

    assert_equal "insert", events.first["headers"]["operation"]
    assert_equal "Hello", events.first["value"]["body"]
  end

  test "no broadcasts when suppressed" do
    assert_no_stream_broadcasts @post do
      Comment.suppressing_streams do
        @post.comments.create!(body: "Silent")
      end
    end
  end
end

Architecture

This gem mirrors turbo-rails 1:1 in structure and design:

Turbo Rails	Durable Streams Rails	Role
Turbo::Streams::StreamName	DurableStreams::Rails::StreamName	Name generation (private stream_name_from)
Turbo::Streams::Broadcasts	DurableStreams::Rails::Broadcasts	Flatten/compact guards, serialization, transport
Turbo::StreamsChannel	DurableStreams module	Entry point (extends StreamName + Broadcasts)
Turbo::Broadcastable	DurableStreams::Rails::Broadcastable	Model concern, delegates to module
Turbo::Streams::ActionBroadcastJob	DurableStreams::Rails::BroadcastJob	Async job, delegates to module
ActionCable::TestHelper	DurableStreams::Rails::Testing	Test transport interception
Turbo::Broadcastable::TestHelper	DurableStreams::Rails::Broadcastable::TestHelper	Test assertions

The key architectural difference: Turbo broadcasts HTML fragments over WebSockets (Action Cable). This gem broadcasts JSON State Protocol events over HTTP (Durable Streams).

Dependencies

• durable_streams — Ruby client for the Durable Streams protocol (resolved automatically via RubyGems)
• railties >= 8.0

Protocol Coverage

The Durable Streams protocol defines 8 operations on streams. This gem handles the server side — broadcasting events, provisioning streams, and authorizing client access via signed URLs. Client-side reads and writes are handled by the official @durable-streams/client JavaScript package, not this gem.

Operations

#	Operation	HTTP	This gem's role
1	Create	PUT	StreamProvisioner — automatic on first use
2	Append	POST	Broadcasts — broadcast_event_to, broadcast_to
3	Read (catch-up)	GET	Authorization only — signed_stream_url
4	Read (long-poll)	GET + live=long-poll	Authorization only — signed_stream_url
5	Read (SSE)	GET + live=sse	Authorization only — signed_stream_url
6	Close	POST + Stream-Closed	Not yet implemented
7	Delete	DELETE	Not yet implemented
8	Metadata	HEAD	Not yet implemented

signed_stream_url generates a signed, expiring URL and provisions the stream. The client uses this URL with @durable-streams/client and chooses the read mode via query parameters — the gem doesn't need to know which mode the client will use.

Access control

Only the server may create streams. The server authenticates with the Durable Streams server via short-lived ES256 JWT Bearer tokens; clients authenticate via signed, expiring tokens generated by this gem.

	Create	Append	Read	Close	Delete	Metadata
Server (this gem)	Yes	Yes	Planned	Planned	Planned	Planned
Client (JS package)	No	Planned	Yes	No	No	Planned

Yes = this gem provides the implementation or authorization today. Planned = permitted by design, not yet implemented. No = not permitted (security decision).

Client append (planned) enables use cases like live cursors and typing indicators where the write payload goes directly to the stream server rather than through a Rails controller. The Durable Streams server verifies ES256 JWTs locally via Caddy's jwtauth on every write to verify the signed token, but this is lightweight — pure HMAC verification with no database or session loading. For reads (SSE), JWT verification happens once at connection time.

Stream provisioning

The Durable Streams protocol requires explicit stream creation (PUT) before any read or write — unlike Action Cable which auto-creates channels on subscribe. StreamProvisioner hides this requirement from application code:

• signed_stream_url calls ensure_stream_exists before generating the URL
• append_to_stream calls ensure_stream_exists before appending
• A Concurrent::Set cache ensures the PUT only fires once per stream name per process

Whichever path touches a stream first provisions it. The application developer never thinks about stream creation.

Security

Two URLs, two authentication channels

The gem separates client-facing and server-to-server communication:

# config/initializers/durable_streams.rb

# Client-facing — used by signed_stream_url to build URLs browsers connect to.
DurableStreams.client_url = ENV.fetch("DURABLE_STREAMS_CLIENT_URL", "http://localhost:4437/v1/streams")

# Server-to-server — used by Rails to POST broadcasts and PUT stream creation.
# Authenticated via short-lived ES256 JWTs (callable Authorization header).
DurableStreams.server_url = ENV.fetch("DURABLE_STREAMS_SERVER_URL", "http://localhost:4437/v1/streams")

In single-server deployments both point to the same address. In multi-server deployments they diverge: client_url is the public domain (behind a CDN), server_url is the internal network address.

server_url is never exposed to clients. client_url is never used for server-to-server communication.

Client authentication — ES256 JWT signed URLs

Browsers receive signed, expiring URLs via Inertia props (or any server-rendered response). The token is an ES256 JWT encoding the stream name, permissions (read/write), issuer, and an expiration timestamp.

Caddy's jwtauth middleware (via ggicci/caddy-jwt) verifies the JWT signature against the public key PEM file. No callback to Rails — verification is entirely local.

The token is domain-agnostic. It validates what (stream name + permissions + expiry), not where (which host the request is hitting). Network-level controls (firewall/NSG rules) must prevent clients from reaching server_url directly.

Server authentication — ES256 JWT Bearer tokens

Rails authenticates to the Durable Streams server with short-lived ES256 JWTs sent as Authorization: Bearer <JWT>. A fresh JWT is minted per-request via a callable lambda in the engine initializer. The private key is stored in Rails credentials.

Future

Brewing secretly — an async-backed durable streams server inside Rails.

License

MIT