SolidLoop

Infrastructure layer for running autonomous AI agents as long-lived Rails background processes. Built on ActiveJob + PostgreSQL with a Rack-style middleware architecture and Model Context Protocol (MCP) tool support.

Each agent turn (LLM call or tool execution) is a discrete background job. State lives in the database — not in memory. Crash the process, restart it, the agent continues from where it stopped.

Installation

Add to your Gemfile (GitHub-only for now — pin a release tag):

gem "solid_loop", github: "ruslan/solid_loop", tag: "v0.0.4"

Then run:

bundle install
bin/rails solid_loop:install:migrations
bin/rails db:migrate

Requires Rails >= 7.1, PostgreSQL, faraday, and event_stream_parser.

Job backend requirement

SolidLoop requires a same-database transactional ActiveJob backend — Solid Queue or GoodJob running in the application database. SolidLoop enqueues each successor job inside the transaction that advances loop state, so state advancement and job dispatch commit atomically: a crash leaves neither a state change without its job nor a job pointing at rows that never committed. (SolidLoop::ApplicationJob sets enqueue_after_transaction_commit = false so the enqueue actually participates in the transaction.)

Non-transactional or external backends (Sidekiq/Resque on Redis, SQS, …) do not get this guarantee: the enqueue escapes the transaction, reintroducing a crash window between state commit and dispatch — and jobs may even run before the enclosing transaction commits. They would need a transactional-outbox layer, which SolidLoop does not provide.

Usage

Define an agent class inheriting from SolidLoop::Base:

class MyAgent < SolidLoop::Base
  def system_prompt
    "You are a helpful assistant."
  end

  def llm_provider
    {
      base_url: "https://api.openai.com/v1",
      api_token: ENV["OPENAI_API_KEY"],
      model: "gpt-4o",         # used in the JSON payload (all dialects)
      # llm_model_name: "...", # used for URL construction in the Gemini dialect
      read_timeout: 120
    }
  end
end

Tools (MCP-only)

Every tool in SolidLoop is an MCP tool. MCP is the interface; where and how a tool server runs is a transport concern. An agent declares its tool servers via mcps — name: is required in every entry:

class MyAgent < SolidLoop::Base
  def mcps
    [
      # 1. Remote MCP server over Streamable HTTP
      { name: :main,
        url: ENV["MCP_URL"],
        api_token: ENV["MCP_TOKEN"],
        tools: %w[list_files read_file],       # optional whitelist
        required_tools: %w[list_files] },       # optional fail-fast validation

      # 2. In-process toolset — the class below IS the MCP server, no HTTP
      { name: :search, toolset: SearchTools },

      # 3. Child-process MCP server over stdio (spawn-per-call, stateless only)
      { name: :fs,
        transport: SolidLoop::Mcp::StdioTransport.new(
          command: ["npx", "-y", "@modelcontextprotocol/server-filesystem", "/data"],
          timeout: 30
        ) }
    ]
  end
end

Toolsets: in-process tools

A toolset groups the tools of one domain into a logical MCP server that runs in-process, on the caller's thread, with full access to your app:

class SearchTools < SolidLoop::Mcp::Toolset
  server_name "search"

  tool "web_search",
       description: "Search the web and return the top results",
       input_schema: {
         type: "object",
         properties: { query: { type: "string" } },
         required: %w[query]
       } do |args, ctx|
    Search::Web.call(loop: ctx.loop, **args.symbolize_keys)
  end

  tool "fetch_url",
       description: "Fetch a URL and return a text summary",
       input_schema: {
         type: "object",
         properties: { url: { type: "string" } },
         required: %w[url]
       } do |args, ctx|
    Search::Fetch.call(loop: ctx.loop, **args.symbolize_keys).to_json
  end
end

The block receives string-keyed args (exactly as they would arrive over HTTP) and a CallContext (ctx.agent, ctx.loop). A String return value becomes text content; a Hash is also exposed as structuredContent; raising produces an isError tool result the model can self-correct on.

Tool names must be loop-unique across servers — add prefix: "search" to an entry to advertise its tools as search__*; collisions without a prefix fail loudly at session initialization. Details — transports, custom adapters, the conformance suite, session_recovery: — in docs/guides/mcp_transports.md; the decision and the migration guide from 0.0.2 native tools in docs/decisions/mcp-only-tooling.md.

Serving toolsets over HTTP (MCP server)

Define a toolset once — the in-process agent consumes it locally, and external MCP clients (Claude Code, other apps) consume the same tools remotely, with shared schemas, wire logs and admin. Mount each toolset you want to expose:

# config/routes.rb
mount SolidLoop::Mcp.server(SearchTools, auth: McpAuth.new) => "/mcp/search"

An unmounted toolset is not exposed. The endpoint is fail-closed: auth: is required and is any object responding to call(token, request) -> principal | nil — return the caller's identity (an API-client record, a user, a symbol) or nil for a 401. Inside tool handlers external calls arrive with ctx.principal set and ctx.agent / ctx.loop nil. Inbound sessions and per-request wire logs appear in the admin UI under Inbound MCP. Details in docs/guides/mcp_transports.md; the design in docs/decisions/mcp-server.md.

Customizing the middleware pipeline per agent

Override configure_llm_middlewares or configure_tool_middlewares to inject middleware into the pipeline for a specific agent only:

class MyAgent < SolidLoop::Base
  # Runs before each LlmCompletionJob — insert/remove LLM pipeline layers here.
  def configure_llm_middlewares(builder)
    builder.insert_before(
      SolidLoop::Middlewares::ResponseParsing,
      SolidLoop::Middlewares::ToolCallXmlParser  # enable XML tool-call parsing for this agent
    )
  end

  # Runs before each ToolExecutionJob — insert/remove tool pipeline layers here.
  def configure_tool_middlewares(builder)
    builder.insert_after(SolidLoop::ToolMiddlewares::ToolExecution, MyAuditMiddleware)
  end
end

Global pipeline changes (applying to all agents) go in an initializer instead — see docs/guides/llm_middlewares.md.

Start a loop (creates DB records and enqueues LlmCompletionJob):

loop_record = MyAgent.start!(
  subject: current_user,
  message: "Summarize this week's engineering updates."
)

Pause and resume:

agent = MyAgent.new(loop_record)
agent.pause!
agent.resume!

Loop status lifecycle: init → queued → running → completed / failed / paused.

Admin UI

SolidLoop ships with a mountable admin UI for inspecting loops, messages, tool calls, MCP sessions/tools, events, and a lightweight dashboard.

Mount the engine in your app routes:

# config/routes.rb
mount SolidLoop::Engine => "/solid_loop"

The admin UI is intentionally unauthenticated inside the gem. Protect the mounted path in the host application the same way you would protect Sidekiq Web or GoodJob admin.

Features include:

• loop/message/tool/event browsing
• loop control actions: observe, freeze, unfreeze, force stop
• live loop message updates with Turbo when observe mode is enabled

Live observation requires turbo-rails

The "observe" mode broadcasts new messages over Turbo Streams, so it depends on turbo-rails being present in the host application. turbo-rails is not a hard dependency of the gem — everything else works without it.

If a loop has observe mode enabled but turbo-rails is not available, SolidLoop skips the live broadcast and logs a warning (Rails.logger.warn) instead of raising. To use live observation, add gem "turbo-rails" to your host app.

Streaming

Each agent controls streaming via the streaming? method. This is a conscious architectural decision, not a default setting.

class MyAgent
  def streaming?
    true  # or false
  end
end

When to use streaming (true)

Scenario	Reasoning
Live chat with a user	Tokens appear as they arrive — no frozen screen for 5–30s
Long generations (>5s) with a frontend	Perceived latency drops dramatically
Slow local model (Ollama, vLLM, llama.cpp)	Each token resets the read timeout — no need for a 2-hour timeout on the full generation
Measuring TTFT	Time-to-first-token is only observable in streaming mode
Cancellation	CancellationError only fires mid-stream

When to use non-streaming (false)

Scenario	Reasoning
Background agents with no UI	No one is watching — streaming adds complexity for no benefit
Batch processing / ETL / cron jobs	You need throughput, not perceived latency
Structured output / JSON mode	Simpler to parse a complete response than partial chunks
Production without a performance dashboard	Removes SSE parsing overhead; simpler failure modes
Short generations (<2s)	Latency difference is imperceptible to users

Rule of thumb: if a human is watching a screen, use streaming. If it's a background job, don't.

Slow local models: streaming wins

With a slow CPU/GPU model, streaming is almost always the right call even for background agents:

• read_timeout is counted between chunks, not over the full response. Each token resets the timer, so a 120s timeout is safe even for a 30-minute generation.
• Without streaming, read_timeout must cover the entire generation — potentially hours. Any network hiccup silently kills the job.

The one exception: if the model runs in a completely isolated environment with no network risk and no UI, non-streaming with a generous timeout is simpler.

Anthropic and Gemini streaming

SolidLoop's built-in streaming parser (SseStreamAggregator) handles OpenAI-compatible SSE format. Anthropic and Gemini use different wire formats — their native streaming will not parse correctly with the anthropic or gemini dialects today.

Solution: use an LLM proxy that presents any provider as an OpenAI-compatible endpoint, then use the open_ai dialect with streaming: true:

Proxy	Notes
LiteLLM	Open-source, self-hosted. Translates 100+ providers to OpenAI format. Recommended for production.
OpenRouter	Hosted service. Single API key, pay-per-token across providers. Good for prototyping.

# Anthropic Claude via LiteLLM proxy — full streaming support
class MyAgent < SolidLoop::Base
  def llm_dialect_name = :open_ai   # proxy speaks OpenAI SSE

  def streaming? = true

  def llm_provider
    {
      base_url:  "http://localhost:4000",   # LiteLLM proxy
      api_token: ENV["LITELLM_API_KEY"],
      model:     "anthropic/claude-opus-4-5"
    }
  end
end

The native anthropic and gemini dialects are still useful for non-streaming production use (background agents, batch jobs) where you control the provider directly without an extra hop.

Read Timeout

read_timeout controls how long to wait for data from the LLM before aborting. It is set per-agent via llm_provider and defaults to 10 minutes.

def llm_provider
  {
    base_url: "...",
    api_token: "...",
    model: "...",
    read_timeout: 300  # seconds
  }
end

How it works: the timeout counts silence between consecutive reads from the socket — not total request duration. During streaming, each incoming token resets the timer. The dangerous window is prefill (before the first token), when the model is silent.

Choosing the right value

Scenario	Recommended
Cloud provider (OpenAI, Gemini, etc.) with streaming	60–120s — prefill is fast, network is reliable
Cloud provider without streaming	120–300s — full response must arrive in one read
Local model with streaming	600–1800s — prefill on CPU can take minutes
Local model without streaming	3600s or more — entire generation (prefill + decode) must complete before the first byte arrives

Rule of thumb: with streaming enabled, set the timeout to cover your worst-case prefill time. Without streaming, it must cover the entire generation — for a large local model producing thousands of tokens on CPU, that can be hours.

Durability: lease + reaper

Every LLM turn is claimed with an execution token (the generation fence) and a lease (lease_expires_at). The lease is derived per claim from the agent's read_timeout plus a margin. When a worker dies mid-turn, the lease expires and a periodic reaper heals the loop.

The LLM heartbeat (why the lease alone isn't enough)

read_timeout is an inactivity timeout — the silence between two socket reads — not a bound on total request duration (see Read Timeout above). A healthy model can stream for tens of minutes with every inter-chunk gap under read_timeout, and pre-call MCP session init is likewise unbounded by it. So a static read_timeout + margin lease would expire on a live, healthy worker mid-turn and be falsely reclaimed.

To prevent that, a lightweight heartbeat renews the lease for the entire claimed LLM job — MCP init + the whole stream (including a silent prefill) + finalization — on a dedicated DB connection, at ~lease/4 cadence. Each renew is conditional on the loop still being running with the same execution token, so it can never renew a lease it no longer owns; if it can't (the generation rotated away or the reaper reclaimed the turn), the worker raises SolidLoop::LostLease before writing anything and yields the turn. Correctness ultimately rests on the commit-time fence check under the row lock; the heartbeat only keeps a healthy turn from being falsely reclaimed.

Tool leases

Each unresolved tool call carries its own per-tool lease (lease_token / leased_until) so the reaper can reclaim one dead parallel tool without fencing its healthy siblings (parallel tools are the shipped default). Unlike the LLM path there is no tool heartbeat — a tool invocation is a single bounded client call, so the per-claim lease derivation suffices:

• The tool lease is derived as max(default_tool_timeout, slowest resolved HTTP MCP client timeout) + lease_margin. An HTTP MCP client's timeout (mcp_config[:timeout] || 60, operator-settable) is folded in, so a healthy long-running HTTP tool is never reclaimed mid-call.
• Residual — in-process / custom-transport tools: these expose no client timeout to derive from, so default_tool_timeout is their ceiling. An in-process tool that legitimately runs longer than default_tool_timeout MAY be reclaimed → a duplicate invocation. The both-token fence keeps SolidLoop's own record clean — exactly one canonical result is committed regardless — but the tool's external effect is at-least-once: the second invocation's side effect DOES run and will repeat unless the handler atomically deduplicates on the stable idempotency key (solid_loop:tool_call:<id>, delivered in-process via CallContext#idempotency_key and on the wire as MCP _meta). So set default_tool_timeout above your slowest in-process tool to avoid the duplicate entirely, and make irreversible handlers dedupe on that key.

Install the reaper recurrence with whatever scheduler you already run — the gem does not own a scheduler. SolidLoop::ReaperJob is a plain entry point for SolidLoop.reap!:

# good_job (config/application.rb or an initializer)
config.good_job.enable_cron = true
config.good_job.cron = {
  solid_loop_reaper: { cron: "* * * * *", class: "SolidLoop::ReaperJob" }
}

SolidLoop.reap! is idempotent and safe to run concurrently (each case re-checks its condition under a row lock, so two racing reapers resolve to one). It re-enqueues dead LLM turns (expired lease), reclaims stalled tool calls, repairs dropped enqueues, re-enqueues loops stuck queued past a threshold, and sweeps orphaned processing shells off paused/terminal loops. SolidLoop.last_reaped_at exposes the timestamp of the last successful reap as a liveness health signal (process-local — see the known limitations below).

Required queues (allowlisting hosts must consume ALL THREE)

SolidLoop routes its jobs onto three fixed queues. A host that runs a worker with an explicit queue allowlist (rather than "all queues") MUST consume every one of them, or work silently stalls:

Queue	Jobs	Consequence if not consumed
solid_loop_llm	LlmCompletionJob (LLM turns)	loops never advance
solid_loop_tool	ToolExecutionJob (tool calls)	tool phases never resolve
solid_loop_maintenance	ReaperJob (recovery sweep)	hard-kill recovery is lost — dead turns/tools are never reclaimed, and reaper jobs accumulate unrun

The ReaperJob queue name (solid_loop_maintenance) is a permanent, stable contract. For example, a GoodJob worker that allowlists queues must include all three:

# config/environments/production.rb (GoodJob example)
config.good_job.queues = "solid_loop_llm,solid_loop_tool,solid_loop_maintenance"
# (or simply "*" to consume every queue)

Tunables (all optional):

SolidLoop.configure do |c|
  c.config.lease_margin          = 60   # seconds added on top of read_timeout / tool timeout when deriving a lease (must be > 0)
  c.config.queued_reap_threshold = 300  # a loop queued longer than this is re-enqueued (lost queue row)
  c.config.default_read_timeout  = 600  # fallback read_timeout when an agent's llm_provider omits one
  c.config.default_tool_timeout  = 600  # ceiling for an in-process/custom tool's lease (raise above your slowest such tool)
  c.config.lease_leak_grace      = 300  # grace (s) added on top of an agent's max_duration to form the renewer's leak ceiling — a leak-safety backstop; must be > 0
  c.config.default_max_duration  = 7200 # fallback max_duration (s) sizing the leak ceiling when a registration omits one (mirrors SolidLoop::Base#max_duration)
end

The invariant lease_margin > 0 guarantees lease > read_timeout (LLM) and lease > tool timeout (tools) per claim; the heartbeat then keeps a healthy LLM turn's lease alive for its full duration. The default_read_timeout knob is the single coherent source for BOTH the derived LLM lease and the HTTP client's fallback read timeout, so the two can never diverge.

The LLM lease renewer (one shared background thread)

The heartbeat is a single process-wide background service (SolidLoop::LeaseRenewer), not one thread per turn. Each in-flight LlmCompletionJob registers its (loop_id, execution_token, lease_duration) on start and deregisters when the turn ends; the renewer batch-renews every registered lease each tick from one long-lived DB connection. This keeps renewal's connection demand O(1) — a single checkout for all in-flight turns — regardless of worker concurrency. (A per-turn renewer thread each checking out its own connection would livelock at full pool occupancy: every heartbeat would block waiting for a connection and treat the exhaustion as a lost lease.)

Because the renewer holds one connection for the process lifetime, size your DB pool as worker_concurrency + 1 so a full set of busy workers plus the renewer never contends. The renewer draws from the same pool as the workers (so it shares their exact view of the data).

Leak safety does not depend on the job's ensure ever running: each registration carries a hard ceiling tied to the owning agent's own legal turn bound — max_duration + lease_leak_grace (default 2h + 5min). Once a registration exceeds it — only possible if a turn somehow never deregistered — the renewer drops it and stops renewing, so its lease lapses and the reaper reclaims the loop. Crucially, a legitimate long turn is never dropped: the agent already caps a legal turn at max_duration (its own contract), so any turn within that bound finishes before the ceiling. (Tying the ceiling to lease_duration × K instead would be wrong — a healthy turn stays registered for its whole life, so any legitimate stream longer than K lease-widths, e.g. a small read_timeout with steady chunks, would be dropped and reaper-reclaimed mid-flight, and no long turn could ever complete.)

Known limitations (0.0.x)

The durability guarantees above are the shipped contract; the following are accepted, documented boundaries for the 0.0.x line (recovery is at-least-once, and canonical DB effects are deduped by the executed_at guard + stable ToolCall key):

• (a) Continuation failure after a tool checkpoint surfaces late. A continuation failure immediately after a tool checkpoint may surface only after the reaper redelivers the checkpointed call — the tool body is not repeated; only the dropped continuation is retried.
• (b) Slow in-process/custom tools may duplicate their external effect. An in-process or custom-transport tool that runs longer than default_tool_timeout may be reclaimed and re-invoked, so its external side effect can run more than once (the canonical DB result is still exactly one). Mitigated by the stable solid_loop:tool_call:<id> idempotency key; raise default_tool_timeout above your slowest such tool.
• (c) Tool-lease derivation bounds configured HTTP MCP calls only. Cumulative setup/recovery time and opaque custom transports can exceed the derived tool lease and be redelivered under the at-least-once contract.
• (d) Reaper case-2 continuation can enqueue duplicates. A dropped-continuation re-enqueue may deliver a duplicate ToolExecutionJob; canonical effects are deduped (idempotent response_message check), so it is harmless.
• (e) SolidLoop.last_reaped_at is process-local, not fleet-wide. It reflects only the reap runs of the current process. For fleet liveness, alert on your scheduler/backend metrics (e.g. GoodJob cron execution), not this value.
• (f) Mixed old/new workers across the lease upgrade are unsupported. Drain jobs or deploy atomically — do not run pre-lease and post-lease workers against the same database simultaneously.
• (g) Lease timestamps are Rails UTC :datetime. Do not reinterpret lease_expires_at / leased_until as local time.
• (h) Reaper scans/indexes are adequate for 0.0.x, not scale-tuned. The partial indexes and per-loop scans are correct and cheap at current scale; large-fleet tuning is an additive later change.
• (i) Middleware that REPLACES a canonical persistence finalizer forfeits the guarantees. Custom middleware that replaces (rather than wraps) a canonical finalizer (ResponseParsing / ToolExecution / ResponseCreation) must reproduce the heartbeat / both-token / row-lock fences itself, or it loses the durability guarantees.
• (j) A hand-rolled paused→running resume MUST revoke the gated tool_call's lease before re-enqueuing. If your host implements its own resume that re-enqueues a ToolExecutionJob for an unexecuted tool_call, it must FIRST NULL that call's lease_token/leased_until (as well as rotate a fresh execution_token) — otherwise the re-enqueued job's claim CAS matches zero rows and no-ops silently, stranding the loop until the stale lease's TTL expires and the reaper reclaims it. SolidLoop::Base#resume! does exactly this (revoke-then-enqueue in one locked txn) and is the pattern to mirror. The claim no-op now logs a ToolExecutionJob: tool lease claim lost … warning so the strand is diagnosable.

Note (streaming UI): mid-stream partials are stored on a hidden message row until the turn commits, so a reclaimed/cancelled/paused partial never re-enters history. pause / force-stop / freeze (and a reaper backstop) mark any orphaned processing shell failed+hidden. To render live streaming in a host view, include hidden processing assistant shells in your live query (they are still broadcast).

Testing

The test suite uses RSpec, WebMock, and FactoryBot against a PostgreSQL dummy app. Tests run against Rails 7.1, 7.2, and 8.1 via appraisals.

Setup

bundle install
bundle exec appraisal install

Create the test database (uses DB_HOST, DB_USERNAME, DB_PASSWORD env vars, defaulting to localhost / postgres / ""):

bundle exec rails db:create db:migrate RAILS_ENV=test

Running tests

# Against the default Gemfile (Rails 8.1)
bundle exec rspec

# Against a specific Rails version
bundle exec appraisal rails-7-1 rspec
bundle exec appraisal rails-7-2 rspec
bundle exec appraisal rails-8-1 rspec

# All versions
bundle exec appraisal rspec

Test infrastructure

StreamingLlmEmulator — stubs the LLM HTTP endpoint (OpenAI-compatible). Matches requests by the last message's content and returns pre-configured streaming (SSE) or non-streaming responses. Records all requests for assertion.

let(:emulator) { StreamingLlmEmulator.new("http://llm.local", "my-token") }

before { emulator.set_answer(/hello/i, { role: "assistant", content: "Hi!" }) }

McpEmulator — stubs an MCP server. Handles initialize / tools/list / tools/call JSON-RPC methods with session validation.

let(:mcp) { McpEmulator.new("http://mcp.local") }

before { mcp.set_tool_result("my_tool", "tool output") }

TestAgent — a minimal SolidLoop::Base subclass defined in the dummy app. Configure it via loop_record.state:

loop_record = TestAgent.start!(
  subject: nil,
  message: "Hello",
  base_url:  "http://llm.local",
  api_token: "my-token",
  model:     "test-model",
  streaming: false
)

License

The gem is available as open source under the terms of the MIT License.