ruby-test-ide

A browser IDE for Ruby whose intelligence comes from running your code and asking the live objects what they can do — the same trick irb uses — plus argument/return types observed while your unit tests run, instead of static analysis or type-annotation files (sorbet/RBS).

t = 'test'     # evaluated for real, in a sandboxed worker process
t.st           # completion asks the actual String instance:
               #   -> start_with?  strip  strip!   (with arity + owner)

Run it

gem install ruby-test-ide
cd your/ruby/project
ruby-test-ide                 # prints e.g. http://localhost:3000/?token=<random> — open that
ruby-test-ide -p 4000 -w path/to/project   # or pick port/workspace

The server executes arbitrary code from any request it accepts, so a random access token is required by default — copy the full URL it prints (including ?token=...); the page then attaches that token to every API call itself, so you won't need to touch the URL again for that session. --no-auth disables the token for CI/automation; --token yourvalue pins a fixed one.

From this repo (development):

ruby exe/ruby-test-ide --workspace workspace   # demo workspace (prints its own token)
rake test        # unit tests (pure helpers, worker round-trip, auth gate)
rake e2e         # browser suite (Playwright, isolated fixture workspace, --no-auth)
rake build       # pkg/ruby-test-ide-x.y.z.gem
rake install     # build + install locally
rake release     # tag vX.Y.Z, push, publish to rubygems (maintainer only)

Zero runtime dependencies (TCPServer, RDoc/ri and TracePoint are stdlib); ruby >= 3.2, verified on 3.2.2 and 3.4.9 — syntax diagnostics understand both parser generations' error formats (3.4's Prism messages are richer and surface as better squiggles).

The sidebar shows a collapsible folder tree (state persists across reloads) of Ruby files (*.rb, Gemfile, Rakefile — not a general file browser, so opening something never force-parses non-Ruby content as Ruby), is resizable by dragging its right edge, opens files, ＋ creates them, Cmd/Ctrl+S saves; unsaved buffers keep their edits when you switch files and are marked with a dot. All paths are fenced inside the workspace root (traversal rejected server-side). An empty directory is seeded with a two-file example on first boot.

What you get in the editor:

Goal (CLAUDE.md)	How it works here
Autocomplete of all methods + parameter counts	On `.`, the buffer up to the cursor is evaluated in a worker; the receiver expression is `eval`'d against that binding and its live methods are listed with `Method#parameters`-derived signatures, arity and owner class. Works for chains (`t.strip.rev`), literals (`[1,2].su`), hash elements (`person[:name].cap`), user-defined methods, locals, constants, keywords.
Ruby docs on hover	Hovering a method looks up the real `ri` database in-process (`RDoc::RI::Driver`, memoised, ~0.15 ms warm). Hovering a variable shows its class and current value.
Error highlighting	Syntax errors (compile-only check, never executed) and runtime errors (from live eval, with the failing line) become Monaco squiggles.
`irb`-grade experience	Better: with Live eval on, every edit re-runs the buffer under a `TracePoint`, so each line gets a grey debugger annotation of the locals it changed (`# t: String = "test"`), and the side panel shows the final state of every variable + captured stdout.
Go to definition	Cmd/Ctrl+click or F12 on any method jumps to its `Method#source_location` — the runtime answer, so it works for methods defined dynamically. Workspace files open in place; gem/core locations are shown in the status bar.
Multi-file workspace	Buffers evaluate in place: the worker chdirs to the file's directory and shims `require_relative`, so `main.rb` can require `lib/greeter.rb` and completion on a `Greeter` instance lists methods defined in the other file — signatures included, unsaved edits included.
Gems via Gemfile	If the workspace has a `Gemfile`, the worker activates it (`bundler/setup`) before evaluating, so `require 'rest-client'` resolves exactly as under `bundle exec` and completion/hover work on gem objects (`RestClient.` → `get(url, headers = …, &block)`). Run `bundle install` with the same ruby the server uses; a missing gem surfaces as a friendly error in the panel.
Types from any test framework or script	Learn from tests runs the commands from `.ruby_ide.yaml` (or auto-detected test files) with a type observer preloaded via `RUBYOPT` into every ruby process they start. Works with minitest, rspec, rake, cucumber, or a plain script — the IDE records, per method, the argument/return classes actually seen with sample values (`.ruby-ide/observations.json`, merged across commands and subprocesses). Completion falls back to this oracle when live eval can't produce the object — `w.current('x').` completes as `Hash` even though `current` sits on top of a mocked network call — and hover shows "Observed in unit tests: current(city: String) → Hash (e.g. ...) — 3 calls".
Access control	A random token (Jupyter-style) is required on every route except `/health` and the vendored static assets, since this server executes whatever code it's sent. Checked via an `X-Ruby-Test-Ide-Token` header (attached automatically by the page once loaded) or a `?token=` query param on first load; comparison is constant-time. `--no-auth` / `--token` override it.

npx playwright test --project=chromium runs the end-to-end suite (autocomplete, live state, diagnostics) against a real browser; the config boots its own server against an isolated fixture workspace automatically.

Architecture

browser (lib/ruby_test_ide/public/index.html, Monaco vendored — no internet needed)
   │  POST /api/complete /hover /diagnostics /run   (JSON)
   ▼
lib/ruby_test_ide/server.rb — zero-gem TCPServer HTTP server
   │  • splits "line before cursor" into receiver expression + prefix
   │    (backward scanner: balanced brackets, strings, chains, ranges)
   │  • syntax check via RubyVM::InstructionSequence.compile (never executes)
   │  • ri doc lookup + memoised cache (RDoc::RI::Driver in-process,
   │    `ri` CLI fallback)
   │  spawns per request ──────────────┐
   ▼                                   │ stdin: one JSON request
lib/ruby_test_ide/runner.rb — worker   │ fd 3:  one JSON reply (user stdout
   • fresh method-body binding         │        can't corrupt the protocol)
     (self == main, no leaked locals)  │ pgroup + kill on 4s timeout
   • eval(code_before, bind) — largest │ rlimit_cpu belt-and-braces
     syntactically-valid prefix, keeps
     partial state when code raises mid-way
   • completion: eval(receiver).methods + Method#parameters
   • run: TracePoint(:line) captures locals per line = debugger view

Design decisions that came out of research into irb/repl_type_completor:

fd 3 for results, child process group, hard kill — user code that puts, forks, or loops forever cannot corrupt or hang the IDE.
Method-body binding as sandbox — TOPLEVEL_BINDING.eval('binding') leaks the host script's locals into completions; a def ...; binding; end binding is clean and keeps script semantics (self is main).
:line TracePoint events fire before the line runs — state shown for line N is taken from the event for line N+1 (and from the binding itself after eval returns, for the last line).
C-implemented core methods report [[:rest]] (gsub(*args)) — real parameter names for those come from the ri call-seq shown on hover. (Method#parameters is fully informative for methods written in Ruby, which includes gem code.)
ENV['BUNDLER_VERSION'] is pinned before bundler/setup — otherwise a lockfile BUNDLED WITH an older installed bundler makes setup re-exec the worker, whose stdin (the request) is already consumed.
Monaco's word pattern is overridden for Ruby — the default splits start_with? before the ?, so hover/definition would query a method that doesn't exist. Receiver evaluation for hover/completion also carries its own 2s timeout, falling back to test observations, so hovering a chain built on a slow network call stays responsive.
Monaco is vendored, not CDN-loaded (lib/ruby_test_ide/public/vendor/monaco, MIT-licensed, its own LICENSE ships alongside) — the IDE works fully offline after gem install. Assets are served with a 1-year immutable cache, safe only because index.html is templated at request time to cache-bust every vendored URL with ?v=<gem version> — an upgrade can't serve a stale cached Monaco. A worker built from a blob: URL resolves importScripts() against its own opaque origin, not the page's, so that URL must be absolute (window.location.origin), not root-relative.
The token is only ever embedded into index.html after that request's own token already checked out — / requires ?token=... like every other route, so there's no window where an unauthenticated request could see it. HTTP header names are case-insensitive but not hyphen-insensitive: X-RubyTestIDE-Token downcases to x-rubytestide-token (no internal hyphens), which silently failed against a hand-written x-ruby-test-ide-token check — caught by a test that reads the real header name out of index.html and drives it through Server#route, rather than hand-copying the same literal on both sides.

Docs prerequisite

Hover docs need an ri database. rvm rubies often ship without one; generate it once (~20 s) with:

cd ~/.rvm/src/ruby-3.2.2 && rdoc --all --ri \
  --op ~/.rvm/rubies/ruby-3.2.2/share/ri/3.2.0/system .

Gems installed with the --no-document default have no ri data; add gem: --document=ri to ~/.gemrc, or backfill a gem with rdoc --ri --op "$(gem spec NAME doc_dir 2>/dev/null || echo $GEM_HOME/doc/NAME-VERSION)/ri" $GEM_HOME/gems/NAME-VERSION/lib.

Honest limitations (it's a PoC)

It executes your code. That is the point — but treat it like running ruby file.rb: file/network side effects happen (worker is killed after 4 s, though, so infinite loops are fine). A "Live eval" toggle turns continuous execution off.
Completion inside an unfinished def/block falls back to the largest parseable prefix, so method-local variables aren't visible until the block closes (irb has the same blind spot mid-edit).
Each request pays worker spawn cost (~50 ms) plus a full re-run of the buffer — fine for scripts, wrong for slow/side-effectful programs.

How "learn from tests" works

Configure it in workspace/.ruby_ide.yaml — any executable counts, because the observer doesn't run your tests, it rides along in them:

learn:
  - bundle exec rspec
  - bundle exec rake test
  - ruby scripts/exercise_everything.rb
timeout: 60          # seconds per command

POST /api/learn
  server: read .ruby_ide.yaml learn: commands
          (fallback: guess from test/**/*_test.rb and spec/)
  each command runs from the workspace root with
          RUBYOPT="-r .../observer.rb"  RUBY_IDE_OBS_DIR  RUBY_IDE_OBS_ROOT
  observer (framework-agnostic, inherited by every ruby subprocess):
          TracePoint(:call, :return) filtered to workspace paths
          → per method: {args: {name: {Class: sample}}, returns: {Class: sample}, calls}
          → dumped to one JSON file per process at exit
  server: merge dumps → workspace/.ruby-ide/observations.json
completion/hover workers read the observations file:
  - receiver eval fails → trailing method name → most-seen return class
    → complete on that class's instance_methods (flagged "observed in tests")
  - hover merges live Method#parameters with observed types + sample values;
    methods with no live object still get a full hover from observations

Mocks are the mechanism, not a problem: tests stub the network edge (def @weather.fetch_json(_) = SAMPLE), so everything above the stub runs for real and the recorded types are real. The observer registers its at_exit before minitest/rspec autorun hooks, so it dumps after the suite finishes; paths like <internal:hash> are excluded so core methods don't pollute the data. Note: learn: commands are arbitrary shell commands run with your privileges — the same trust level as the IDE evaluating your code. Ruby 3.4 note: minitest is a bundled gem, so it must be listed in the workspace Gemfile to load under bundler.

Where this goes next

Persist + cache context (as CLAUDE.md suggests): keep a warm worker per buffer, snapshot line-states, invalidate only below the edited line.
Wrap it as an LSP server — the runner already produces everything textDocument/completion|hover|publishDiagnostics need, which would make the same engine drive VS Code natively as an extension.
Hybrid inference for un-run code paths: seed types from the live binding, then abstract-interpret the current expression the way repl_type_completor (irb's opt-in type completor) does — no execution of the hovered chain, RBS-aware, still runtime-grounded.
Deeper project awareness: the file tree, per-file buffers and cross-file requires exist now; next is rdoc generation for the user's own lib/ (hover docs for their classes), file rename/delete, and evaluating entry points other than the open buffer (e.g. run tests to learn types on code paths the buffer doesn't reach).