gem nosj

gem nosj is an very fast JSON parser and generator for Ruby, written in Rust on the first-party nosj crate and SIMD-accelerated on every platform (NEON on Apple Silicon, SSE2/AVX2 on x86-64).

gem nosj is the powerful evil twin of the json gem.

GitHub Release Docs

  • It is faster than gem json and every third-party parser, including Oj, RapidJSON, FastJsonparser, Yajl. 1.0–1.8× faster than the bundled json gem, 1.3–11× faster than Oj, and up to 17× faster than Yajl—see Benchmarks.
  • It has lazy documents: NOSJ.lazy wraps a document and parses a value only when you touch it—repeated access costs nanoseconds, and everything you never read is never parsed.
  • It has a partial parsing mode: JSON Pointer lookups that pull single values out of big documents in microseconds, skipping everything else.
  • It has file APIs: parse, generate, dig, and lazy-wrap files directly—no throwaway file-sized Ruby String, and the partial modes memory-map the file so unread pages never even leave the disk.
  • It does byte-splicing edits: NOSJ.splice changes a field inside a passing payload 10–51× faster than parse-mutate-generate, and RFC 6902 JSON Patch / RFC 7386 merge patch apply straight to the raw string.
  • It is great to debug with: parse errors carry line, column, and a caret snippet pointing at the break, and NOSJ.stats X-rays a mystery blob (depth, value counts, key histogram) faster than parsing it.
  • It accelerates a Rails application in both encoding and decoding.
  • It comes precompiled (platform gems built with per-platform optimizations, nothing to compile on install).
  • Otherwise, same API and option names as gem json.

And there's more: validate documents without building a single Ruby object, minify and pretty-print without parsing, resolve whole batches of paths in one pass, stream NDJSON / JSON Lines in both directions, and accelerate an entire application with a one-line drop-in.

Requirements

  • Ruby 3.3 or newer (CRuby; tested on 3.3, 3.4, and 4.0).
  • Linux (x86-64 and arm64, glibc and musl), macOS (Apple Silicon), or Windows (x64): these platforms install precompiled, per-platform-optimized gems with nothing to build.

Getting started

bundle add nosj

require "nosj"

NOSJ.parse('{"a":[1,true]}')       #=> {"a" => [1, true]}
NOSJ.generate({"a" => [1, true]})  #=> '{"a":[1,true]}'

That's it—if you know the json gem, you already know nosj.

gem json compatibility

Want the speedup without touching your code? One line reroutes JSON.parse, JSON.generate, JSON.pretty_generate, and JSON.dump through nosj:

require "nosj/json"

In a Bundler app (Rails included) that can live entirely in the Gemfile you can do this:

gem "nosj", require: "nosj/json"

A MultiJson adapter ships too:

require "nosj/multi_json"

And then MultiJson.use NOSJ::MultiJsonAdapter.

Ruby on Rails

In a Rails app, use this for "Rails mode":

gem "nosj", require: "nosj/rails"

That installs a nosj-backed ActiveSupport JSON encoder, so obj.to_json, render json:, and ActiveSupport::JSON.encode walk the object tree natively: values that aren't JSON-native recurse through as_json exactly like ActiveSupport's own encoder, non-finite floats encode as null, and HTML-safety escaping (escape_html_entities_in_json) behaves identically—verified differentially against ActiveSupport's encoder. It loads the drop-in too, so ActiveSupport::JSON.decode and JSON request-body parsing ride the fast path.

Measured against ActiveSupport's own encoder: ×1.9 on small documents up to ×5.2 on large trees and ×14 on HTML-heavy content—see Benchmarks → Rails mode.

What's in the box

json API

The json gem API, on the NOSJ module:

NOSJ.parse(src, symbolize_names: true)   # also: freeze, max_nesting,
                                         # allow_nan, allow_trailing_comma
NOSJ.generate(obj)                       # indent, space, object_nl, ...,
NOSJ.pretty_generate(obj)                # ascii_only, script_safe, strict

Lazy documents

NOSJ.lazy wraps a document in a lazy view: read a field and only that path is parsed—containers stay lazy, scalars arrive as plain Ruby values, and repeated reads are cached:

doc = NOSJ.lazy(json)
doc["users"][3]["name"]   # parses only this path
doc.dig("meta", "count")  # a whole path in one fused resolution
doc["users"].size         # counted without materializing anything
doc["users"][3].value     # materialize one subtree (parse options apply)

Pass a frozen string and creating the view is practically free— nanoseconds, even on a megabyte document. Malformed content raises when it is first read, not at wrap time.

Partial parsing

Pull values out of a document without materializing the rest—skipped content is stepped over at SIMD block speed, so a lookup costs what it skips, not what the document weighs:

NOSJ.dig(json, "users", 3, "name")        # Hash#dig-shaped
NOSJ.at_pointer(json, "/users/3/name")    # JSON Pointer

# Many lookups in one pass. A batch costs about as much as its
# single deepest member:
NOSJ.at_pointers(json, ["/users/3/name", "/meta/count"])
NOSJ.dig_many(json, [["users", 3, "name"], ["meta", "count"]])

Example: an early field resolves in ~0.35µs where JSON.parse(json).dig(...) costs ~980µs on the same document—three orders of magnitude. A field at the far end of a 570 KB document costs ~71µs, still 13× faster than parse-then-dig. Misses return nil; matched subtrees materialize with the same options as parse (symbolize_names:, freeze:).

Files API

Every mode has a file-native form, so a document never round-trips through a throwaway Ruby String:

NOSJ.load_file("config.json")                 # 1.3× File.read + parse
NOSJ.write_file("out.json", obj)              # generate straight to disk
NOSJ.dig_file("huge.json", "users", 3, "name")   # never reads the rest
NOSJ.at_pointer_file("huge.json", "/meta/count")
doc = NOSJ.load_lazy_file("huge.json")        # lazy view over a memory map

The partial and lazy forms memory-map the file, so pages you never read are never loaded from disk. Missing files raise the usual Errno exceptions. Measured numbers live in Benchmarks → File APIs.

Reformat without parsing

NOSJ.minify and NOSJ.reformat pipe the parser's events straight into the emission kernels: zero Ruby objects are built for the document (the specs assert it), SIMD in and SIMD out. Minifying the 631 KB twitter.json takes 409µs—3.4× faster than a generate(parse(x)) round-trip, 3.9× faster than gem json's cycle, and only 1.4× the cost of valid?:

NOSJ.minify(json)                        # compact
NOSJ.reformat(json, pretty: true)        # pretty_generate layout
NOSJ.reformat(json, ascii_only: true)    # escape-transcode, no parse
NOSJ.reformat_file("big.json")           # straight off a memory map

Output is exactly generate(parse(json))—canonical number spellings, normalized escapes, same formatting options—except duplicate keys pass through (a reformatter must not silently drop data) and lone-surrogate strings re-escape instead of raising. Acceptance options apply too: minify(src, allow_trailing_comma: true) normalizes the commas away.

Byte-splicing edits and JSON Patch

Change a field in a passing payload without the parse → mutate → generate cycle: NOSJ.splice skips to the target spans (one pass for the whole batch) and rebuilds the string around them—every byte outside the targets is copied untouched, so formatting, key order, and number spellings survive exactly. On twitter.json this is 10×–51× faster than parse-mutate-generate (129µs for a late field, 26µs for an early one, vs ~1.3ms):

NOSJ.splice(json, "/config/timeout" => 30)
NOSJ.splice(json, "/a" => 1, "/b/c" => [true])   # batch, one pass

On top of the same machinery: RFC 6902 JSON Patch applied to the raw string (add, remove, replace, move, copy, test—the whole appendix-A suite is in the specs), and RFC 7386 JSON Merge Patch:

NOSJ.patch(json, [
  {"op" => "test", "path" => "/a", "value" => 1},
  {"op" => "replace", "path" => "/a", "value" => 2},
  {"op" => "add", "path" => "/list/-", "value" => "x"}
])
NOSJ.merge_patch(json, {"config" => {"legacy" => nil, "timeout" => 30}})

Missing splice targets raise KeyError (use patch add to insert); failed patches raise NOSJ::PatchError; inserted values are byte-identical to NOSJ.generate's output.

NDJSON / JSON Lines

Logs, data pipelines, LLM batch files—one JSON document per line. NOSJ.each_line streams values out (1.6× faster than the line-split-and-JSON.parse idiom on twitter statuses), and NOSJ.generate_lines builds the whole stream in one buffer pass (4.1× faster than map-generate-join):

NOSJ.each_line(log) { |event| ingest(event) }   # or an Enumerator:
NOSJ.each_line(log).first(10)                   # walks only 10 lines

NOSJ.generate_lines(events)          #=> %({"a":1}\n{"b":2}\n...)
NOSJ.each_line_file("events.ndjson") { |e| }    # over a memory map
NOSJ.write_lines("out.ndjson", events)          # straight to disk

Blank lines are skipped, one value per line is enforced, and a malformed line raises a rich ParserError whose #line is the physical line in the stream. Parse options apply per line—NOSJ.each_line(log, freeze: true) yields Ractor-shareable values.

Validation without parsing

NOSJ.valid? runs the full parser—tokenizers, string decode, number validation—into a null sink and allocates no Ruby objects at all. It is 2-4× faster than NOSJ.parse:

NOSJ.valid?('{"a":1}')                #=> true
NOSJ.valid?('{"a":}')                 #=> false
NOSJ.valid?(src, max_nesting: false)  # same options as parse

Document statistics

NOSJ.stats answers "what is this 40 MB blob": one counting pass through the same null-sink machinery—no Ruby value is built for the document, and it costs less than a parse (~1.3× faster on twitter.json):

s = NOSJ.stats(blob)          # or NOSJ.stats_file("huge.json")
s[:byte_size]                 #=> 631514
s[:root]                      #=> :object
s[:max_depth]                 #=> 10
s[:values]                    #=> {total: 13914, objects: 1264, arrays: 1050,
                              #    strings: 4754, integers: 2108, ...}
s[:keys]                      #=> {total: 13345, unique: 94}
s[:key_histogram].first(3)    #=> [["id", 447], ["id_str", 447], ["urls", 364]]
s[:containers]                #=> {max_object_entries: 40, max_array_length: 100}
s[:strings]                   #=> {bytes: 200716, max_bytes: 463}

Nesting is unlimited by default (a deep blob is exactly what a diagnostic should describe); malformed documents raise the usual rich ParserError.

Rich parse errors

A failed parse raises NOSJ::ParserError carrying where the document broke—#byte_offset, #line, #column, and a caret #snippet—computed only when a parse fails, so success pays nothing. Positions stay absolute through partial parsing, lazy documents, and the file APIs; unrescued errors print the snippet:

NOSJ.parse(%({\n  "a": 1,\n  "b": }))
# NOSJ::ParserError: unexpected character at byte 19
#   e.line     #=> 3
#   e.column   #=> 8
#   e.snippet  #=>   "b": }
#                         ^

Benchmarks

Every installed JSON gem, benchmark-ips: AWS EC2 c7a.2xlarge (AMD EPYC 9R14, Zen 4), Ruby 4.0.6 + YJIT, json 2.21.1, Oj 3.17.4, RapidJSON 0.4.0, FastJsonparser 0.6.0, Yajl 1.4.3, PGO build, 2026-07-16. ×N = times slower than nosj.

Parse

file nosj (i/s) json Oj FastJsonparser RapidJSON Yajl
activitypub 12.4k ×1.18 ×1.53 ×1.89 ×1.99 ×4.57
canada 248 ×1.10 ×8.28 ×1.45 ×1.51 ×4.96
citm_catalog 504 ×1.03 ×1.97 ×2.07 ×1.83 ×4.93
gsoc-2018 397 ×1.30 ×1.47 ×1.81 ×1.80 ×4.78
homebrew-formula 15.2 ×1.13 ×1.54 ×2.42 ×2.02 ×4.30
homebrew-llvm 49.3k ×1.60 ×1.48 ×1.50 ×1.85 ×5.11
mesh 1.1k ×1.30 ×3.93 ×1.82 ×1.95 ×6.96
numbers 5.9k ×1.19 ×4.64 ×1.43 ×1.75 ×7.04
ohai 14.4k ×1.47 ×1.61 ×2.11 ×1.97 ×4.71
simple 977k ×1.31 ×1.77 ×2.08 ×1.63 ×5.15
small_mixed 3.2M ×1.48 ×2.85 ×2.44 ×1.82 ×6.75
tolstoy 8.9k ×1.79 ×1.96 ×2.29 ×2.10 ×17.31
twitter 1.1k ×1.09 ×1.83 ×2.25 ×2.61 ×5.40

Generate

file nosj (i/s) json Oj RapidJSON Yajl
activitypub 34.8k ×1.20 ×1.78 ×2.41 ×5.84
canada 150 ×0.98* ×10.98 ×11.20 ×10.87
citm_catalog 1.3k ×1.04 ×1.44 ×1.55 ×2.85
gsoc-2018 1.1k ×1.28 ×2.64 ×3.64 ×11.00
homebrew-formula 20.9 ×1.07 ×1.25 ×1.62 ×3.03
homebrew-llvm 71.5k ×1.23 ×2.22 ×3.04 ×5.80
mesh 613 ×1.08 ×9.24 ×9.40 ×9.26
numbers 2.1k ×1.03 ×10.63 ×11.00 ×10.69
ohai 39.4k ×1.07 ×1.29 ×1.50 ×3.37
simple 2.2M ×1.03 ×1.58 ×1.62 ×4.36
small_mixed 5.6M ×1.09 ×2.26 ×1.83 ×7.16
tolstoy 8.3k ×1.30 ×4.15 ×6.50 ×14.54
twitter 2.9k ×1.09 ×1.46 ×2.01 ×4.05

* canada-generate is a statistical tie with the json gem (within measurement error).

File APIs

twitter.json (570 KB) from a warm page cache, medians of 7 alternating rounds against the plain-Ruby composition on the same parser (Apple Silicon dev box, Ruby 4.0.6 + YJIT, PGO build, 2026-07-16):

operation µs/op vs the Ruby way
NOSJ.load_file (parse the whole file) 948 ×1.33 vs NOSJ.parse(File.read(path))
NOSJ.dig_file (one deep field) 246 ×5.2 vs read + parse + dig
NOSJ.load_lazy_file + one field 257 ×5.0 vs read + parse + dig

NOSJ.write_file measures at parity with File.write(NOSJ.generate(obj)) on this box—file-write timings swing too much for an honest multiplier; what it saves is the intermediate file-sized Ruby String.

Rails mode

ActiveSupport::JSON.encode with the nosj encoder installed, against stock ActiveSupport (Apple Silicon dev box, Ruby 4.0.6 + YJIT, activesupport 8.1, medians of 5 interleaved per-process rounds, outputs verified byte-identical first, 2026-07-17; rake bench:rails):

workload nosj (i/s) vs ActiveSupport
twitter tree (570 KB) 4.5k ×5.2
100-record index (with timestamps) 33.7k ×1.7
HTML-heavy user content 177.7k ×14.2
Time/Date/BigDecimal hash 1.1M ×3.0
small API hash 4.2M ×1.9
small hash to_json 4.1M ×1.9

The HTML-safety escaping that dominates stock encodes of user-generated content is fused into the SIMD string-emission kernels here: escaped output costs the same single pass as unescaped.

Reproduce with rake bench (the parity-gated comparison, after a PGO retrain—the shipping configuration) or rake bench:ips (the multi-gem shoot-out).

Switching from the json gem

You mostly don't have to do anything. Some differences:

  • The legacy object-deserialization options (create_additions, object_class, array_class, decimal_class) raise ArgumentError; the nosj/json drop-in falls back to the original gem for them.
  • Behaviors the json gem itself deprecates (JS comments, raw invalid UTF-8) follow the strict semantics instead.
  • Unlike Array#dig, negative indices in NOSJ.dig return nil (JSON Pointer has no equivalent).
  • Parse errors raise NOSJ::ParserError (NOSJ::NestingError past max_nesting, like the gem); messages use byte offsets rather than the gem's phrasing, and the exception carries #line, #column, and a caret #snippet.

Everything else—including the gem's exact float formatting, which is not the shortest-round-trip form most libraries emit—matches byte-for-byte and is verified continuously against the full corpus.

How it works

Most fast parsers build their own tree first and convert it into Ruby objects second, paying for every string and container twice. nosj is built on the nosj Rust crate, an event parser with no tree of its own:

  • No intermediate tree. The crate parses with NEON/SSE2/AVX2 SIMD kernels and emits events; the extension builds interned hash keys, strings, and containers directly on Ruby's heap during the parse, with GC-safe value stacks and epoch-evicted key caches. Generation walks Ruby objects once, streaming through fused scan-and-store escape kernels.
  • Byte-exact floats. Output reproduces the json gem's fpconv (Grisu2) float format digit for digit—round-tripping is verified, not assumed.
  • PGO everywhere. Local builds, CI, and every precompiled platform gem train on the benchmark corpus before the shipping compile; the precompiled binaries use portable codegen with SIMD tiers detected at runtime.

Development

bundle exec rake compile                # build the extension (applies a PGO profile if present)
bundle exec rake spec                   # the gem-parity suite
bundle exec rake bench                  # PGO retrain + the parity-gated sweep vs the json gem
bundle exec rake bench:fast             # the sweep without retraining
bundle exec rake "bench:ips[twitter]"   # multi-gem shoot-out (benchmark-ips); no args = full corpus

Acknowledgements

Thanks to Jean Boussier, Florian Frank, Hiroshi Shibata, Nobuyoshi Nakada, Étienne Barrié, and the other authors and maintainers of the json gem—for their work on the gem itself, for optimization ideas, and for some of the JSON documents in the benchmark corpus.

Thanks to Mat Sadler for magnus.

Thanks to the authors of the projects credited in NOTICE and the crate's NOTICE: fpconv and Florian Loitsch's Grisu2, fast_float and fast_double_parser (Daniel Lemire et al.), simdjson (Daniel Lemire, Geoff Langdale, and contributors), and the architectural lineage of yyjson, simd-json, and sonic-rs.

Assisted by

Claude Fable 5, Claude Opus 4.8.

License

MIT. The underlying Rust crate is MIT AND BSL-1.0 AND Apache-2.0; its NOTICE file itemizes the derived components.