json5-ruby

json5-ruby is a Pure Ruby JSON5 1.0.0 parser. Its syntactic parser is generated from grammar/json5.y with Ibex 0.3.0; Ibex and ibex-runtime are development/build dependencies only.

Installation

gem "json5-ruby"

Usage

require "json5"

JSON5.parse(<<~JSON5)
  {
    unquoted: 'value',
    hex: 0x2a,
    values: [true, null,],
  }
JSON5
# {"unquoted"=>"value", "hex"=>42, "values"=>[true, nil]}

The parser accepts comments, single-quoted strings, trailing commas, IdentifierName keys, hexadecimal numbers, Infinity, and NaN. Input must be a valid UTF-8 or US-ASCII Ruby String; other encodings are not implicitly transcoded.

JSON5.parse returns Ruby values by default. Use number_mode: :float or :lexeme, string_mode: :code_units for exact UTF-16 code units, and duplicate_keys: :first, :error, or :preserve when the default last-value policy is not appropriate. Lone surrogates in values require code-unit mode or parse_document; duplicate_keys: :preserve also retains lone-surrogate member names as JSON5::ECMAString keys because Ruby UTF-8 strings cannot represent them losslessly.

Set warn_duplicate_keys: true to emit duplicate_key diagnostics for duplicates discarded by the :last or :first policies.

document = JSON5.parse_document("{ /* keep me */ answer: 0x2a, }")
document.root.raw
# "{ /* keep me */ answer: 0x2a, }"

Document nodes retain byte spans, raw lexemes, decoded code units, and leading/trailing trivia. Member nodes expose trivia before : and around the comma/closing separator; array nodes expose the corresponding element separator trivia. The source is copied and frozen by default.

Limits and diagnostics

JSON5::Limits.default protects input size, nesting depth, string and number length, comment length, and container entries. Pass limits: JSON5::Limits.unbounded only for trusted input. Diagnostics can be collected in an array or sent to a callable; by default warnings are sent to Warning.warn. Unescaped U+2028 and U+2029 in strings produce diagnostics.

The defaults are 64 MiB input, 512 nesting levels, 16 Mi UTF-16 string code units, 1 MiB number tokens, 16 MiB comments, and 10,000,000 entries per container. Construct JSON5::Limits with individual keyword overrides when an application needs a different boundary; a nil field disables that one limit.

warnings = []
JSON5.parse("'line\u2028separator'", diagnostics: warnings)
warnings.first.code
# "unescaped_line_separator_in_string"

Development

bundle install
bundle exec rake
bundle exec rake generated
bundle exec rake conformance
bundle exec rake fuzz
bundle exec rake security
bundle exec rake package

bundle exec rake release:check runs all deterministic release gates above. The generated and conformance tasks are read-only: intentional updates use bundle exec rake generated:update and bundle exec rake conformance:update, followed by the corresponding read-only check. rake generated regenerates into a temporary directory, runs Ibex's own --check against that raw output, then compares the canonicalized parser, manifest, automaton, verifier report, metrics, and Unicode table byte-for-byte with the committed artifacts. The package gate verifies that runtime dependencies are empty, inspects gem contents, installs into a clean temporary gem home, and smoke-tests the installed artifact.

Performance thresholds are evaluated only between results captured on the same fixture, Ruby runtime, and machine. Capture before/after files with benchmark/run.rb, then run an explicit threshold gate:

ruby benchmark/run.rb --iterations=25 --warmup=5 --output=before.json
# apply the candidate change, using the same host and Ruby
ruby benchmark/run.rb --iterations=25 --warmup=5 --output=after.json
bundle exec rake "benchmark:check[before.json,after.json,5,5]"

The final two arguments are the maximum allowed median-runtime and allocation regression percentages. The comparison requires identical fixture/result checksums, Ruby/OS/CPU/compiler/host metadata, warmup, batch, and iteration settings, YJIT state, and relevant environment knobs; incompatible or invalid results are rejected. GitHub-hosted CI uploads a report-only artifact because those runners do not provide a stable performance environment. Maintainers run the Benchmark Gate workflow at the immutable benchmark-builder-v12 tag with a reviewed full candidate SHA. The base is fixed by benchmark/baseline.commit; callers cannot substitute it. benchmark/trusted_builder.commit pins the workflow revision that supplies the trusted harness, comparator, manifest, and fixtures on a one-job ephemeral + json5-benchmark self-hosted runner. It covers value, lexer, document, and error paths, uses predeclared balanced before/after process rounds, enforces both 5% thresholds on every round and their aggregate, and checks 1x/2x/4x linearity. Never attach the label to a persistent runner: candidate parser code is executed, so the runner must be isolated, one-time, and kept on a stable power/performance profile. The runner must be Linux, provide Bubblewrap at /usr/bin/bwrap, and provide Ruby from /opt/hostedtoolcache. Candidate code runs without network or inherited credentials in isolated user, mount, PID, IPC, UTS, cgroup, and network namespaces; its host inputs are read-only and only an ephemeral /tmp is writable.

The workflow artifact is a tar archive containing aggregate evidence.json, a SHA-256 file manifest, and all raw measurements. A separate GitHub-hosted job revalidates it and creates a GitHub artifact attestation bound to the pinned builder workflow. The signed contents bind the exact candidate release commit. Before release, set JSON5_BENCHMARK_EVIDENCE to the downloaded .tar.gz file; the release task verifies that attestation before it accepts the fixed base, exact release/harness commit, complete manifest coverage, passing comparisons, and passing linearity evidence. This verification requires GitHub CLI network access.

release:check also refuses uncommitted or untracked files, so commit intended generated evidence before running the final release gate.

The generated parser, Automaton IR, verification report, metrics, grammar, source manifest, fixed Unicode 3.0.0 input, and conformance matrix are checked in. See docs/conformance-matrix.md for the coverage map and the design/work-plan documents under .idea/ for the implementation boundary and release checklist.

Security issues should be reported privately as described in SECURITY.md. Conformance scope and known limitations are recorded in docs/conformance-report.md.

Compatibility

The public API is JSON5; Json5 remains an alias for the initial gem scaffold. The supported Ruby version is 3.2 or newer. This project does not execute input as Ruby code and never converts member names to symbols.

License

MIT. See LICENSE.txt.