Module: MockServer::LLM

Defined in:
lib/mockserver/llm.rb

Overview

Idiomatic Ruby LLM-mocking builder API for MockServer.

This module mirrors the Java/Node/Python client LLM builders (LlmMockBuilder, LlmConversationBuilder, TurnBuilder, LlmFailoverBuilder) and the underlying server-side model classes (Completion, ToolUse, Usage, StreamingPhysics, EmbeddingResponse).

The builders produce plain Ruby Hashes with camelCase string keys that serialise to exactly the same expectation wire JSON the other clients emit. The expectation action is carried in the httpLlmResponse field (a sibling of httpRequest, scenarioName, scenarioState, newScenarioState, times, timeToLive, httpResponse). Nil fields are omitted (NON_NULL).

Examples:

A single completion mock

MockServer::LLM.llm_mock('/v1/chat/completions')
  .with_provider(MockServer::LLM::Provider::OPENAI)
  .with_model('gpt-4o')
  .responding_with(MockServer::LLM.completion.with_text('Hello!'))
  .apply_to(client)

Defined Under Namespace

Modules: Provider, Role Classes: Completion, EmbeddingResponse, IsolationSource, LlmConversationBuilder, LlmFailoverBuilder, LlmMockBuilder, RawExpectation, StreamingPhysics, ToolUse, TurnBuilder, Usage

Constant Summary collapse

SCENARIO_PREFIX =

LlmConversationBuilder — multi-turn conversation with scenario state.

'__llm_conv_'
ISOLATION_MARKER = 
'__iso='
DONE_STATE = 
'__done'

Class Method Summary collapse

Class Method Details

.build_llm_response(provider, model, completion, embedding, conversation_predicates, chaos) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



421
422
423
424
425
426
427
428
429
430
 
# File 'lib/mockserver/llm.rb', line 421

def self.build_llm_response(provider, model, completion, embedding, conversation_predicates, chaos)
  omit_nil(
    'provider' => provider,
    'model' => model,
    'completion' => wire(completion),
    'embedding' => wire(embedding),
    'conversationPredicates' => wire(conversation_predicates),
    'chaos' => wire(chaos)
  )
end

.completion ⇒ Completion

Returns:



334
335
336
 
# File 'lib/mockserver/llm.rb', line 334

def self.completion
  Completion.new
end

.conversation ⇒ LlmConversationBuilder

Entry point mirroring LlmConversationBuilder.conversation().

Returns:



712
713
714
 
# File 'lib/mockserver/llm.rb', line 712

def self.conversation
  LlmConversationBuilder.new
end

.cookie(name) ⇒ IsolationSource

Returns:



411
412
413
 
# File 'lib/mockserver/llm.rb', line 411

def self.cookie(name)
  IsolationSource.new('cookie', name)
end

.default_error_body(status_code) ⇒ Object

LlmFailoverBuilder — N failures then a success completion.



719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
 
# File 'lib/mockserver/llm.rb', line 719

def self.default_error_body(status_code)
  type, message =
    case status_code
    when 429
      ['rate_limit_error', 'Rate limit exceeded. Please retry after a brief wait.']
    when 500
      ['internal_server_error', 'An internal error occurred. Please retry your request.']
    when 502
      ['bad_gateway', 'Bad gateway. The upstream server returned an invalid response.']
    when 503
      ['service_unavailable', 'The service is temporarily overloaded. Please retry later.']
    else
      ['error', "Request failed with status #{status_code}"]
    end
  JSON.generate('error' => { 'type' => type, 'message' => message })
end

.embedding ⇒ EmbeddingResponse

Returns:



377
378
379
 
# File 'lib/mockserver/llm.rb', line 377

def self.embedding
  EmbeddingResponse.new
end

.header(name) ⇒ IsolationSource

Returns:



401
402
403
 
# File 'lib/mockserver/llm.rb', line 401

def self.header(name)
  IsolationSource.new('header', name)
end

.input_tokens(count) ⇒ Usage

Returns:



146
147
148
 
# File 'lib/mockserver/llm.rb', line 146

def self.input_tokens(count)
  Usage.new.with_input_tokens(count)
end

.jitter(amount) ⇒ StreamingPhysics

Returns:



229
230
231
 
# File 'lib/mockserver/llm.rb', line 229

def self.jitter(amount)
  StreamingPhysics.new.with_jitter(amount)
end

.llm_failover ⇒ LlmFailoverBuilder

Entry point mirroring LlmFailoverBuilder.llmFailover().

Returns:



851
852
853
 
# File 'lib/mockserver/llm.rb', line 851

def self.llm_failover
  LlmFailoverBuilder.new
end

.llm_mock(path) ⇒ LlmMockBuilder

Entry point mirroring LlmMockBuilder.llmMock(path).

Returns:



499
500
501
 
# File 'lib/mockserver/llm.rb', line 499

def self.llm_mock(path)
  LlmMockBuilder.new(path)
end

.omit_nil(hash) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Build a Hash from the given pairs, omitting nil values.



47
48
49
50
51
 
# File 'lib/mockserver/llm.rb', line 47

def self.omit_nil(hash)
  result = {}
  hash.each { |k, v| result[k] = v unless v.nil? }
  result
end

.output_tokens(count) ⇒ Usage

Returns:



151
152
153
 
# File 'lib/mockserver/llm.rb', line 151

def self.output_tokens(count)
  Usage.new.with_output_tokens(count)
end

.post_matcher(path) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



416
417
418
 
# File 'lib/mockserver/llm.rb', line 416

def self.post_matcher(path)
  { 'method' => 'POST', 'path' => path }
end

.query_parameter(name) ⇒ IsolationSource

Returns:



406
407
408
 
# File 'lib/mockserver/llm.rb', line 406

def self.query_parameter(name)
  IsolationSource.new('query_parameter', name)
end

.streaming_physics ⇒ StreamingPhysics

Returns:



219
220
221
 
# File 'lib/mockserver/llm.rb', line 219

def self.streaming_physics
  StreamingPhysics.new
end

.time_to_first_token(value, time_unit = 'MILLISECONDS') ⇒ Hash

Delay representing time-to-first-token: { ‘timeUnit’ => .., ‘value’ => .. }.

Returns:

  • (Hash) 


235
236
237
 
# File 'lib/mockserver/llm.rb', line 235

def self.time_to_first_token(value, time_unit = 'MILLISECONDS')
  { 'timeUnit' => time_unit, 'value' => value }
end

.tokens_per_second(count) ⇒ StreamingPhysics

Returns:



224
225
226
 
# File 'lib/mockserver/llm.rb', line 224

def self.tokens_per_second(count)
  StreamingPhysics.new.with_tokens_per_second(count)
end

.tool_use(name = nil) ⇒ ToolUse

Returns:



101
102
103
 
# File 'lib/mockserver/llm.rb', line 101

def self.tool_use(name = nil)
  ToolUse.new(name)
end

.usage ⇒ Usage

Returns:



141
142
143
 
# File 'lib/mockserver/llm.rb', line 141

def self.usage
  Usage.new
end

.validate_status_code(status_code) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



737
738
739
740
741
 
# File 'lib/mockserver/llm.rb', line 737

def self.validate_status_code(status_code)
  if status_code < 100 || status_code > 599
    raise ArgumentError, "statusCode must be between 100 and 599, got #{status_code}"
  end
end

.wire(value) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Convert a value to its wire form: call to_h on builder objects, leave Hashes/scalars untouched.



56
57
58
59
60
61
62
 
# File 'lib/mockserver/llm.rb', line 56

def self.wire(value)
  return nil if value.nil?
  return value.map { |v| wire(v) } if value.is_a?(Array)
  return value.to_h if value.respond_to?(:to_h) && !value.is_a?(Hash)

  value
end