dsprb

Speech signal processing and acoustic feature extraction in pure Ruby. No dependencies, no native extensions, no pseudo-random number generator anywhere.

Extracted from taiwancards, where it turns learner recordings into the feature sequences its pronunciation scorer compares.

The problem

A waveform is a poor representation for comparison. Two recordings of the same syllable differ in amplitude, in microphone response, in speaking rate, and in every sample value, while agreeing on everything a listener perceives: the pitch contour, the resonances of the vocal tract, the noise spectrum of the fricatives.

Extracting those invariants is the work of acoustic phonetics, and it decomposes into a small set of classical algorithms. This library implements them directly: framing and windowing, the discrete Fourier transform, mel filterbanks and the cepstrum, the YIN period estimator, linear prediction and its all-pole envelope, spectral moments, and the filters that condition a signal beforehand.

Install

gem "dsprb"

Waveforms

DSP::Wav decodes 8, 16, 24 and 32 bit PCM as well as 32 and 64 bit IEEE float, including WAVE_FORMAT_EXTENSIBLE, and downmixes multi-channel streams to mono. Everything downstream takes a DSP::Waveform, so the sample rate travels with the samples and cannot be mismatched by accident.

require "dsprb"

signal = DSP.read("utterance.wav")

signal.length       # => 16000
signal.sample_rate  # => 16000.0
signal.duration     # => 1.0
signal.nyquist      # => 8000.0

Signals synthesized in memory are ordinary arrays:

signal = DSP::Waveform.new(samples: samples, sample_rate: 16_000)

Fundamental frequency

DSP::Yin implements the YIN estimator: squared difference over lag, cumulative mean normalization, absolute-threshold period selection, and parabolic refinement of the chosen minimum. The signal is decimated to a working rate before the search, which is what makes an O(window × lag) method affordable in Ruby.

track = DSP.pitch(signal)

track.length        # => 120
track.hop_seconds   # => 0.008
track.voiced_ratio  # => 1.0
track.voiced_f0     # => [196.08, ...]

track.each_voiced { |frequency, time, confidence| ... }

Unvoiced frames report 0.0 rather than nil, so a track is always a dense numeric sequence. Confidence is 1 - d'(τ), the depth of the selected minimum.

Cepstral features

DSP::Mfcc composes a power spectrum, a mel filterbank and a DCT-II. The filterbank supports vocal tract length warping, which rescales the band centers to compensate for speaker anatomy.

frames = DSP.mfcc(signal)

frames.length      # => 98
frames.first.length # => 13

The pieces stay separately usable, so a log-mel spectrogram costs one call:

spectrum = DSP::Spectrum.new(512)
filterbank = DSP::MelFilterbank.new(sample_rate: 16_000, size: 512, filters: 26, warp: 1.05)

log_mel = signal.frames(window: 400, hop: 160).map do |frame|
  filterbank.log_energies(spectrum.power(DSP::Window.apply(frame, DSP::Window.hamming(400))))
end

Resonances

DSP::Lpc fits an all-pole model by the Levinson-Durbin recursion and exposes the spectral envelope it implies. DSP::Formants picks the envelope maxima, refines them parabolically on the log magnitude, and merges peaks that a single resonance split in two.

DSP.formants(vowel)
# => [752.8, 2335.8, 3695.6, 5416.1]

DSP::Formants.new.peaks(vowel).first.amplitude
# => 2.91

The predictor itself is available when the envelope matters more than the peaks:

model = DSP::Lpc.solve(frame, order: 10)

model.order              # => 10
model.error              # => residual energy of the prediction
model.envelope(points: 512)

Spectral moments

Centroid, spread, skewness and excess kurtosis of the power spectrum over a chosen band. These are the standard descriptors of fricative noise, where place of articulation shifts the centroid.

moments = DSP::SpectralMoments.of(power, sample_rate: 16_000, size: 1024, low: 500.0)

moments.centroid  # => 4904.8
moments.spread    # => 1064.8
moments.skewness  # => 0.742
moments.kurtosis  # => -0.708

Energy, filters, resampling

DSP::Energy.frame_db(frame)                                  # root mean square, in decibels
DSP::Energy.zero_crossing_rate(frame)                        # crossings per sample interval
DSP::Energy.envelope_db(samples, window: 400, hop: 160)      # sliding energy, one value per hop

DSP.highpass(signal, cutoff: 60.0)                           # removes rumble and DC offset
DSP.lowpass(signal, cutoff: 4000.0, sections: 2)             # cascaded Butterworth sections
DSP.decimate(signal, 4)                                      # windowed-sinc anti-aliasing

DSP::Curve.resample(contour, 16)                             # contours onto a common grid
DSP::Scales.hz_to_semitones(392.0, reference: 196.0)         # => 12.0

In practice

taiwancards teaches Taiwanese Mandarin and scores learner recordings against native templates. Its analysis stage is a pipeline over this library.

Conditioning. The recording is high-passed to strip rumble and any DC offset, then the energy envelope and zero crossing rate locate the speech boundaries and the syllable onsets.

Tone. Mandarin lexical tone is the fundamental frequency contour, so DSP.pitch carries the primary signal rather than a secondary cue. The track is cleaned of octave errors, then DSP::Curve.resample brings contours of different duration onto a fixed 16-point grid where the four tones become comparable shapes.

Timbre. Thirteen MFCC per frame describe the spectral envelope independently of pitch. Vocal tract length warping is set per speaker, so a child and an adult producing the same vowel land in the same region of the feature space.

Vowel quality. The first two formants place a vowel on the F1/F2 plane, which is where the contrast between ㄧ, ㄩ and ㄨ actually lives.

Fricatives. Spectral moments separate the sibilants: the retroflex ㄕ and the alveolar ㄙ differ mainly in centroid and skewness, not in duration or energy.

The resulting sequences vary in length because speaking rate varies, so they are aligned and averaged with dtwrb, the companion library for dynamic time warping and barycenter averaging.

Speech is one application. Anything sampled as a numeric signal — vibration, biosignals, sonar, telemetry — uses the same spectral and predictive machinery.

Design

Every stage is an object that precomputes what it can and then freezes: DSP::Fourier::Radix2 caches its twiddle factors, DSP::MelFilterbank its triangular bands, DSP::Decimator its kernel. They hold no mutable state, share nothing globally, and are safe to reuse across threads. Build them once, outside the loop.

Collaborators are injected rather than assumed: the transform behind a DSP::Spectrum, the window function of an MFCC sequence, the filterbank of a DSP::Mfcc. A different FFT backend needs only #call(real, imaginary) and #size.

DSP::Waveform, DSP::PitchTrack, DSP::SpectralMoments, DSP::Biquad, DSP::Lpc::Model and DSP::Formants::Peak are Data value objects: frozen, compared by value, copied with #with.

License

WTFPL — see LICENSE.