Class: Parse::Embeddings::Voyage
- Defined in:
- lib/parse/embeddings/voyage.rb
Overview
Voyage AI embeddings provider. Wraps POST /v1/embeddings for
text-only models and POST /v1/multimodalembeddings for the
multimodal text+image models (text-input path only in v5.0; the
image-input path lands with Provider#embed_image in v5.1).
Supported models:
- v4 family —
voyage-4-large(MoE flagship, Matryoshka-capable),voyage-4,voyage-4-lite,voyage-4-nano(Apache 2.0, open-weight on Hugging Face — also runnable through LocalHTTP when self-hosted on vLLM / Ollama / llama.cpp). - v3 family —
voyage-3-large,voyage-3,voyage-3-lite,voyage-code-3. - domain models —
voyage-finance-2,voyage-law-2. - multimodal —
voyage-multimodal-3(1024-dim). Unified text+image vector space at the network boundary. This provider exposes the text-input path only: routes to/v1/multimodalembeddingswith a{ inputs: [{ content: [{ type: "text", text: … }] }] }envelope. The same model will accept image inputs in v5.1 when theembed_imagehook ships; text vectors stored today will sit in the same space as the eventual image vectors (no re-embed required).
== Asymmetric input types
Voyage's input_type field accepts "query" or "document"
(mapped from the SDK-canonical :search_query / :search_document
Symbols). The values are functionally analogous to Cohere's
search_query / search_document — they're encoded by separately
tuned heads, so re-using one type for both sides of a retrieval
pair measurably degrades recall.
Voyage also accepts null (omit the field), which Voyage's docs
recommend for "general purpose" embeddings unrelated to retrieval.
We translate the absent / non-retrieval cases to null rather
than picking a default — Voyage's training depends on the
asymmetry, so guessing on the caller's behalf would be worse than
passing-through.
== Security
- The Faraday connection refuses
proxy:unless the caller opts in viaallow_faraday_proxy: true. Env-proxy autodiscovery (HTTPS_PROXYetc.) is suppressed by default. #inspect(inherited from Provider) never surfaces@api_key.AuthorizationandVoyage-Api-Keyare in Middleware::BodyBuilder::REDACTED_HEADERS.
Defined Under Namespace
Classes: AuthenticationError, BadRequestError, RateLimitError, TransientError
Constant Summary collapse
- DEFAULT_BASE_URL =
"https://api.voyageai.com/v1"- DEFAULT_MODEL =
"voyage-3"- DEFAULT_TIMEOUT =
30- DEFAULT_OPEN_TIMEOUT =
5- DEFAULT_MAX_RETRIES =
3- DEFAULT_BATCH_SIZE =
Voyage's documented per-request cap is 128 inputs.
128- MAX_RESPONSE_BYTES =
16 * 1024 * 1024
- MODEL_DEFAULT_DIMENSIONS =
Native vector widths per model. The v4 family is Voyage's current flagship line (MoE for
voyage-4-large, open-weight nano under Apache 2.0).voyage-4-largesupports Matryoshka truncation via the constructor'sdimensions:override. { "voyage-4-large" => 2048, "voyage-4" => 1024, "voyage-4-lite" => 512, "voyage-4-nano" => 256, "voyage-3-large" => 1024, "voyage-3" => 1024, "voyage-3-lite" => 512, "voyage-code-3" => 1024, "voyage-finance-2" => 1024, "voyage-law-2" => 1024, "voyage-multimodal-3" => 1024, }.freeze
- MODEL_MAX_INPUT_TOKENS =
{ "voyage-4-large" => 32_000, "voyage-4" => 32_000, "voyage-4-lite" => 32_000, "voyage-4-nano" => 32_000, "voyage-3-large" => 32_000, "voyage-3" => 32_000, "voyage-3-lite" => 32_000, "voyage-code-3" => 32_000, "voyage-finance-2" => 16_000, "voyage-law-2" => 16_000, "voyage-multimodal-3" => 32_000, }.freeze
- MATRYOSHKA_MODELS =
Models that accept Voyage's
output_dimensionMatryoshka truncation parameter. Sending the field for other models is rejected with a 400 by Voyage, so we gate it explicitly. %w[voyage-4-large].freeze
- MULTIMODAL_MODELS =
Models that route to
/v1/multimodalembeddingswith the{ inputs: [{ content: [...] }] }envelope rather than the standard/v1/embeddings{ input: [String] }envelope. Text-only inputs from this provider are wrapped as{ type: "text", text: s }content rows. %w[voyage-multimodal-3].freeze
- INPUT_TYPE_WIRE_VALUES =
Map SDK-canonical input_type symbols to Voyage wire strings.
:classification/:clusteringmap tonil(omitted) since Voyage only distinguishes retrieval halves — other intents should receive the unconditioned vector. { search_query: "query", search_document: "document", classification: nil, clustering: nil, }.freeze
Constants inherited from Provider
Provider::AS_NOTIFICATION_NAME
Instance Method Summary collapse
- #backoff_seconds(attempt) ⇒ Object protected
- #build_connection ⇒ Object protected
-
#build_multimodal_body(strings, wire_input_type) ⇒ Object
protected
Build the wire body for
/v1/multimodalembeddings. -
#build_text_body(strings, wire_input_type) ⇒ Object
protected
Build the wire body for the standard
/v1/embeddingsendpoint (text-only models). - #dimensions ⇒ Object
- #embed_batch_size ⇒ Object
-
#embed_image(sources, input_type: :search_document, allow_insecure: false) ⇒ Array<Array<Float>>
Embed a batch of image URLs through Voyage's
/v1/multimodalembeddingsendpoint. -
#embed_text(strings, input_type: :search_document) ⇒ Array<Array<Float>>
Vectors aligned 1:1 with
strings. -
#extract_vectors!(payload, input_count) ⇒ Object
protected
Voyage's response shape mirrors OpenAI:.
-
#initialize(api_key:, model: DEFAULT_MODEL, base_url: DEFAULT_BASE_URL, timeout: DEFAULT_TIMEOUT, open_timeout: DEFAULT_OPEN_TIMEOUT, max_retries: DEFAULT_MAX_RETRIES, embed_batch_size: DEFAULT_BATCH_SIZE, dimensions: nil, truncation: true, allow_faraday_proxy: false, allow_insecure_base_url: false, connection: nil) ⇒ Voyage
constructor
A new instance of Voyage.
- #inspect_attrs ⇒ Object
- #max_input_tokens ⇒ Object
-
#modalities ⇒ Array<Symbol>
Voyage's multimodal models support
[:text, :image]; text-only models report[:text]. - #model_name ⇒ Object
- #normalize? ⇒ Boolean
- #parse_json_body!(body) ⇒ Object protected
- #post_embeddings(body, path: "embeddings") ⇒ Object protected
- #retry_after_seconds(response) ⇒ Object protected
- #supports_input_type? ⇒ Boolean
Methods inherited from Provider
#embed_text_batched, #inspect, #instrument_embed, #validate_response!
Constructor Details
#initialize(api_key:, model: DEFAULT_MODEL, base_url: DEFAULT_BASE_URL, timeout: DEFAULT_TIMEOUT, open_timeout: DEFAULT_OPEN_TIMEOUT, max_retries: DEFAULT_MAX_RETRIES, embed_batch_size: DEFAULT_BATCH_SIZE, dimensions: nil, truncation: true, allow_faraday_proxy: false, allow_insecure_base_url: false, connection: nil) ⇒ Voyage
Returns a new instance of Voyage.
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 |
# File 'lib/parse/embeddings/voyage.rb', line 155 def initialize( api_key:, model: DEFAULT_MODEL, base_url: DEFAULT_BASE_URL, timeout: DEFAULT_TIMEOUT, open_timeout: DEFAULT_OPEN_TIMEOUT, max_retries: DEFAULT_MAX_RETRIES, embed_batch_size: DEFAULT_BATCH_SIZE, dimensions: nil, truncation: true, allow_faraday_proxy: false, allow_insecure_base_url: false, connection: nil ) validate_api_key!(api_key) validate_model!(model) sanitized_base_url = validate_base_url!(base_url, allow_insecure_base_url) validate_positive_integer!(:timeout, timeout) validate_positive_integer!(:open_timeout, open_timeout) validate_non_negative_integer!(:max_retries, max_retries) validate_positive_integer!(:embed_batch_size, ) if > 128 raise ArgumentError, "Parse::Embeddings::Voyage: embed_batch_size #{} exceeds Voyage's per-request cap (128)." end unless [true, false].include?(truncation) raise ArgumentError, "Parse::Embeddings::Voyage: truncation must be true or false (got #{truncation.inspect})." end validate_dimensions!(model, dimensions) @api_key = api_key @model = model @dimensions = dimensions || MODEL_DEFAULT_DIMENSIONS.fetch(model) @base_url = sanitized_base_url @timeout = timeout @open_timeout = open_timeout @max_retries = max_retries @embed_batch_size = @truncation = truncation @allow_faraday_proxy = allow_faraday_proxy @connection = connection || build_connection end |
Instance Method Details
#backoff_seconds(attempt) ⇒ Object (protected)
547 548 549 |
# File 'lib/parse/embeddings/voyage.rb', line 547 def backoff_seconds(attempt) [0.5 * (2**(attempt - 1)), 30.0].min end |
#build_connection ⇒ Object (protected)
383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 |
# File 'lib/parse/embeddings/voyage.rb', line 383 def build_connection headers = { "Authorization" => "Bearer #{@api_key}", "Content-Type" => "application/json", "Accept" => "application/json", "User-Agent" => "parse-stack-embeddings/#{user_agent_version}", } faraday_opts = { url: @base_url, headers: headers } faraday_opts[:proxy] = nil unless @allow_faraday_proxy conn = Faraday.new(**faraday_opts) do |f| f..timeout = @timeout f..open_timeout = @open_timeout f.adapter Faraday.default_adapter end conn.proxy = nil if !@allow_faraday_proxy && conn.respond_to?(:proxy=) conn end |
#build_multimodal_body(strings, wire_input_type) ⇒ Object (protected)
Build the wire body for /v1/multimodalembeddings. The text
path wraps each input string as a single {type: "text", text:}
content row. Image inputs will land in v5.1 alongside
Provider#embed_image; for now the provider is text-only and
the multimodal envelope's content array always contains a
single text row per input.
432 433 434 435 436 437 438 439 440 441 442 443 |
# File 'lib/parse/embeddings/voyage.rb', line 432 def build_multimodal_body(strings, wire_input_type) body = { inputs: strings.map { |s| { content: [{ type: "text", text: s }] } }, model: @model, } body[:input_type] = wire_input_type if wire_input_type # `truncation` is documented for the multimodal endpoint too — # forward it for parity with the text path so callers get the # same fail-on-overlength behavior across models. body[:truncation] = @truncation body end |
#build_text_body(strings, wire_input_type) ⇒ Object (protected)
Build the wire body for the standard /v1/embeddings endpoint
(text-only models).
405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 |
# File 'lib/parse/embeddings/voyage.rb', line 405 def build_text_body(strings, wire_input_type) body = { input: strings, model: @model, truncation: @truncation, } # Only forward input_type when it has a wire value. Voyage # treats absent and `null` identically (unconditioned head), # but absent is the spec-correct form for non-retrieval intent. body[:input_type] = wire_input_type if wire_input_type # `output_dimension` is only valid for the Matryoshka-capable # models. Forward when the configured model is in the # Matryoshka set and the active dimensions differ from native. # Sending it elsewhere would yield a 400. if MATRYOSHKA_MODELS.include?(@model) && @dimensions != MODEL_DEFAULT_DIMENSIONS.fetch(@model) body[:output_dimension] = @dimensions end body end |
#dimensions ⇒ Object
199 200 201 |
# File 'lib/parse/embeddings/voyage.rb', line 199 def dimensions @dimensions end |
#embed_batch_size ⇒ Object
207 208 209 |
# File 'lib/parse/embeddings/voyage.rb', line 207 def @embed_batch_size end |
#embed_image(sources, input_type: :search_document, allow_insecure: false) ⇒ Array<Array<Float>>
Embed a batch of image URLs through Voyage's
/v1/multimodalembeddings endpoint. v5.1 ships URL-only — the
provider receives a public URL and issues its own fetch. The
SDK does NOT download the image; it validates the URL through
Parse::Embeddings.validate_image_url! (CIDR / port / host
allowlist, sentinel-gated egress opt-in) and forwards the
canonicalized URL string in the { type: "image_url",
image_url: ... } content row.
Multimodal model required. Voyage's text-only models
(voyage-3, voyage-4, etc.) do not accept image inputs;
calling embed_image on a provider configured with one of
those raises BadRequestError before any network call.
Bytes-fetch path is v5.3. A future bytes: option will
download via File.safe_open_url, MIME-sniff the
leading bytes, optionally EXIF-strip, and forward as
base64. URL-only ships first because it sidesteps EXIF /
MIME-confusion class issues entirely.
312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 |
# File 'lib/parse/embeddings/voyage.rb', line 312 def (sources, input_type: :search_document, allow_insecure: false) unless MULTIMODAL_MODELS.include?(@model) raise BadRequestError, "Parse::Embeddings::Voyage#embed_image: model #{@model.inspect} does not " \ "accept image inputs. Configure the provider with a multimodal model " \ "(supported: #{MULTIMODAL_MODELS.inspect})." end unless sources.is_a?(Array) raise ArgumentError, "Parse::Embeddings::Voyage#embed_image expects Array of image URLs " \ "(got #{sources.class})." end return [] if sources.empty? unless INPUT_TYPE_WIRE_VALUES.key?(input_type) raise ArgumentError, "Parse::Embeddings::Voyage#embed_image input_type #{input_type.inspect} not in " \ "#{INPUT_TYPE_WIRE_VALUES.keys.inspect}." end # Voyage caps multimodal requests at the same per-request size # as the text endpoint. The text path goes through # `embed_text_batched` which chunks automatically; the image # path has no chunker yet (every directive is a single URL in # v5.1), so guard the direct-API caller against a silent 400. if sources.length > @embed_batch_size raise ArgumentError, "Parse::Embeddings::Voyage#embed_image: batch size #{sources.length} exceeds " \ "the configured cap #{@embed_batch_size} (Voyage per-request max: 128). " \ "Split the input and call embed_image once per chunk." end # Validate every URL up-front so a malformed entry in slot N # does not get past validation while slots 0..N-1 are already # in the wire body. The validator returns the canonicalized # URL — we forward exactly that, not the caller's raw input. canonical_urls = sources.each_with_index.map do |url, i| unless url.is_a?(String) raise ArgumentError, "Parse::Embeddings::Voyage#embed_image sources[#{i}] is not a String " \ "(#{url.class}). v5.1 ships URL-only — bytes/IO support is v5.3." end Parse::Embeddings.validate_image_url!(url, allow_insecure: allow_insecure) end wire_input_type = INPUT_TYPE_WIRE_VALUES[input_type] body = { inputs: canonical_urls.map { |u| { content: [{ type: "image_url", image_url: u }] } }, model: @model, truncation: @truncation, } body[:input_type] = wire_input_type if wire_input_type (sources.length, input_type, modality: :image) do |emit_payload| payload = (body, path: "multimodalembeddings") if payload.is_a?(Hash) && payload["usage"].is_a?(Hash) tt = payload["usage"]["total_tokens"] emit_payload[:total_tokens] = tt if tt.is_a?(Integer) && tt >= 0 end vectors = extract_vectors!(payload, sources.length) validate_response!(sources.length, vectors) end end |
#embed_text(strings, input_type: :search_document) ⇒ Array<Array<Float>>
Returns vectors aligned 1:1 with strings.
227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 |
# File 'lib/parse/embeddings/voyage.rb', line 227 def (strings, input_type: :search_document) unless strings.is_a?(Array) raise ArgumentError, "Parse::Embeddings::Voyage#embed_text expects Array<String> (got #{strings.class})." end return [] if strings.empty? strings.each_with_index do |s, i| unless s.is_a?(String) raise ArgumentError, "Parse::Embeddings::Voyage#embed_text strings[#{i}] is not a String (#{s.class})." end if s.empty? raise ArgumentError, "Parse::Embeddings::Voyage#embed_text strings[#{i}] is empty; Voyage rejects empty inputs." end end unless INPUT_TYPE_WIRE_VALUES.key?(input_type) raise ArgumentError, "Parse::Embeddings::Voyage#embed_text input_type #{input_type.inspect} not in " \ "#{INPUT_TYPE_WIRE_VALUES.keys.inspect}." end wire_input_type = INPUT_TYPE_WIRE_VALUES[input_type] # Multimodal models route to a different endpoint with a # different request envelope. The response envelope shape is # the same (`{ data: [{ embedding, index }], usage: {...} }`) # so `extract_vectors!` is reused as-is. body = if MULTIMODAL_MODELS.include?(@model) build_multimodal_body(strings, wire_input_type) else build_text_body(strings, wire_input_type) end path = MULTIMODAL_MODELS.include?(@model) ? "multimodalembeddings" : "embeddings" (strings.length, input_type) do |emit_payload| payload = (body, path: path) # Voyage's response carries `usage: { total_tokens }`. if payload.is_a?(Hash) && payload["usage"].is_a?(Hash) tt = payload["usage"]["total_tokens"] emit_payload[:total_tokens] = tt if tt.is_a?(Integer) && tt >= 0 end vectors = extract_vectors!(payload, strings.length) validate_response!(strings.length, vectors) end end |
#extract_vectors!(payload, input_count) ⇒ Object (protected)
Voyage's response shape mirrors OpenAI:
{ "object": "list", "data": [ { "object": "embedding", "embedding": [...], "index": 0 }, ... ], "model": "voyage-3", "usage": { "total_tokens": N } }
513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 |
# File 'lib/parse/embeddings/voyage.rb', line 513 def extract_vectors!(payload, input_count) unless payload.is_a?(Hash) raise InvalidResponseError, "Parse::Embeddings::Voyage: response body is not a JSON object." end data = payload["data"] unless data.is_a?(Array) raise InvalidResponseError, "Parse::Embeddings::Voyage: response.data is not an Array." end if data.length != input_count raise InvalidResponseError, "Parse::Embeddings::Voyage: response.data.length #{data.length} != input count #{input_count}." end sorted = data.each_with_index.map do |entry, i| unless entry.is_a?(Hash) raise InvalidResponseError, "Parse::Embeddings::Voyage: response.data[#{i}] is not a JSON object." end idx = entry["index"] unless idx.is_a?(Integer) && idx >= 0 && idx < input_count raise InvalidResponseError, "Parse::Embeddings::Voyage: response.data[#{i}].index #{idx.inspect} out of range." end [idx, entry["embedding"]] end indices = sorted.map(&:first) if indices.uniq.length != indices.length raise InvalidResponseError, "Parse::Embeddings::Voyage: duplicate index in response.data." end sorted.sort_by(&:first).map(&:last) end |
#inspect_attrs ⇒ Object
377 378 379 |
# File 'lib/parse/embeddings/voyage.rb', line 377 def inspect_attrs super.merge(base: safe_base_host, retries: @max_retries) end |
#max_input_tokens ⇒ Object
211 212 213 |
# File 'lib/parse/embeddings/voyage.rb', line 211 def max_input_tokens MODEL_MAX_INPUT_TOKENS[@model] end |
#modalities ⇒ Array<Symbol>
Returns Voyage's multimodal models support
[:text, :image]; text-only models report [:text].
277 278 279 |
# File 'lib/parse/embeddings/voyage.rb', line 277 def modalities MULTIMODAL_MODELS.include?(@model) ? %i[text image] : [:text] end |
#model_name ⇒ Object
203 204 205 |
# File 'lib/parse/embeddings/voyage.rb', line 203 def model_name @model end |
#normalize? ⇒ Boolean
215 216 217 218 |
# File 'lib/parse/embeddings/voyage.rb', line 215 def normalize? # Voyage's v3 embeddings are documented unit-normalized. true end |
#parse_json_body!(body) ⇒ Object (protected)
489 490 491 492 493 494 495 496 497 498 499 500 |
# File 'lib/parse/embeddings/voyage.rb', line 489 def parse_json_body!(body) s = body.to_s if s.bytesize > MAX_RESPONSE_BYTES raise InvalidResponseError, "Parse::Embeddings::Voyage: response body exceeds #{MAX_RESPONSE_BYTES} bytes " \ "(#{s.bytesize}). Refusing to parse." end JSON.parse(s, max_nesting: 32) rescue JSON::ParserError => e raise InvalidResponseError, "Parse::Embeddings::Voyage: response is not valid JSON (#{e.})." end |
#post_embeddings(body, path: "embeddings") ⇒ Object (protected)
445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 |
# File 'lib/parse/embeddings/voyage.rb', line 445 def (body, path: "embeddings") attempts = 0 loop do attempts += 1 begin response = @connection.post(path) do |req| req.body = body.to_json end rescue Faraday::TimeoutError, Faraday::ConnectionFailed => e if attempts > @max_retries raise TransientError, "Parse::Embeddings::Voyage: #{e.class} after #{attempts} attempt(s)." end sleep(backoff_seconds(attempts)) next end status = response.status return parse_json_body!(response.body) if status >= 200 && status < 300 if status == 401 raise AuthenticationError, "Parse::Embeddings::Voyage: 401 Unauthorized — check api_key." end if status == 429 if attempts > @max_retries raise RateLimitError, "Parse::Embeddings::Voyage: 429 rate limited after #{attempts} attempt(s)." end sleep(retry_after_seconds(response) || backoff_seconds(attempts)) next end if status >= 500 if attempts > @max_retries raise TransientError, "Parse::Embeddings::Voyage: #{status} after #{attempts} attempt(s)." end sleep(backoff_seconds(attempts)) next end raise BadRequestError, "Parse::Embeddings::Voyage: #{status} from POST /#{path}." end end |
#retry_after_seconds(response) ⇒ Object (protected)
551 552 553 554 555 556 |
# File 'lib/parse/embeddings/voyage.rb', line 551 def retry_after_seconds(response) ra = response.respond_to?(:headers) ? response.headers["retry-after"] || response.headers["Retry-After"] : nil return nil unless ra v = ra.to_f v.positive? ? [v, 60.0].min : nil end |
#supports_input_type? ⇒ Boolean
220 221 222 |
# File 'lib/parse/embeddings/voyage.rb', line 220 def supports_input_type? true end |