Module: Ask::Instrumentation

Defined in:
lib/ask/instrumentation.rb,
lib/ask/instrumentation/chat.rb,
lib/ask/instrumentation/tool.rb,
lib/ask/instrumentation/version.rb,
lib/ask/instrumentation/embedding.rb

Overview

Instrumentation for LLM observability in the ask-rb ecosystem.

Provides a thin wrapper around ActiveSupport::Notifications for emitting and subscribing to LLM events such as chat completions, embeddings, tool calls, and image generation.

Events follow the {operation}.ask naming convention:

chat.ask        # Chat completion
chat.stream.ask # Streaming chat
tool.ask        # Tool execution
embedding.ask   # Embedding generation
image.ask       # Image generation

Each event payload includes provider-agnostic keys such as provider, model, duration, and provider-specific metrics (input_tokens, output_tokens, etc.).

Usage

require "ask/instrumentation"

# Subscribe to all ask events
Ask::Instrumentation.subscribe do |event|
  puts "#{event.name}: #{event.duration}ms"
end

# Subscribe with a callable subscriber
Ask::Instrumentation.subscribe(MySubscriber.new)

# Instrument a chat completion
Ask::Instrumentation.instrument("chat.ask", provider: "openai", model: "gpt-4") do
  # your LLM call here
end

# Wrap with metadata context
Ask::Instrumentation.with_metadata(user_id: 42, session_id: "abc") do
  Ask::Instrumentation.instrument("chat.ask", provider: "openai", model: "gpt-4") do
    # ...
  end
end

Defined Under Namespace

Modules: Chat, Embedding, Tool

Constant Summary collapse

VERSION = 
"0.1.0"

Class Method Summary collapse

Class Method Details

.current_metadata ⇒ Hash

Return the current thread’s metadata hash.

This is the metadata hash that will be merged into any event payload emitted via instrument in the current thread.

Examples:

Ask::Instrumentation.current_metadata # => {}
Ask::Instrumentation.with_metadata(user_id: 42) do
  Ask::Instrumentation.current_metadata # => { user_id: 42 }
end

Returns:

  • (Hash) —

    The current metadata (empty hash if none set)



158
159
160
 
# File 'lib/ask/instrumentation.rb', line 158

def current_metadata
  Thread.current[:ask_instrumentation_metadata] || {}
end

.instrument(name, payload = {}) { ... } ⇒ Object

Instrument a block of code, wrapping it in an ActiveSupport::Notifications event.

The event name should follow the {operation}.ask convention. The payload is automatically merged with the current thread metadata (set via with_metadata) before being emitted.

Examples:

Ask::Instrumentation.instrument("chat.ask", provider: "openai", model: "gpt-4") do
  # LLM call
end

Parameters:

  • name (String) —

    The event name (e.g. “chat.ask”)

  • payload (Hash) (defaults to: {}) —

    The event payload

Yields:

  • The block to instrument

Returns:

  • (Object) —

    The return value of the block



113
114
115
116
 
# File 'lib/ask/instrumentation.rb', line 113

def instrument(name, payload = {})
  merged = current_metadata.merge(payload)
  ActiveSupport::Notifications.instrument(name, merged) { yield if block_given? }
end

.subscribe(pattern = /\.ask$/, subscriber) ⇒ ActiveSupport::Notifications::Fanout::Subscriber .subscribe(pattern = /\.ask$/, &block) ⇒ ActiveSupport::Notifications::Fanout::Subscriber

Subscribe to ask instrumentation events.

Accepts a callable subscriber (an object that responds to #call) and/or a block. When a pattern is given, only events matching the pattern are delivered to the subscriber.

Examples:

Subscribe with a block

Ask::Instrumentation.subscribe { |event| puts event.name }

Subscribe with a callable object

Ask::Instrumentation.subscribe(MySubscriber.new)

Subscribe with a pattern and callable

Ask::Instrumentation.subscribe(/chat\.ask/, MySubscriber.new)

Overloads:

  • .subscribe(pattern = /\.ask$/, subscriber) ⇒ ActiveSupport::Notifications::Fanout::Subscriber

    Parameters:

    • pattern (Regexp, String) (defaults to: /\.ask$/) —

      Optional event name pattern

    • subscriber (#call) —

      An object that responds to #call(event)

    Returns:

    • (ActiveSupport::Notifications::Fanout::Subscriber)
  • .subscribe(pattern = /\.ask$/, &block) ⇒ ActiveSupport::Notifications::Fanout::Subscriber

    Parameters:

    • pattern (Regexp, String) (defaults to: /\.ask$/) —

      Optional event name pattern

    • block (Proc) —

      The callback invoked on each matching event

    Returns:

    • (ActiveSupport::Notifications::Fanout::Subscriber) 


77
78
79
 
# File 'lib/ask/instrumentation.rb', line 77

def subscribe(pattern = /\.ask$/, subscriber = nil, &block)
  ActiveSupport::Notifications.subscribe(pattern, subscriber || block)
end

.unsubscribe(subscriber_or_name) ⇒ void

This method returns an undefined value.

Unsubscribe from events.

Delegates to ActiveSupport::Notifications.unsubscribe. Accepts either the subscriber object returned from subscribe or a string/regexp pattern.

Examples:

subscriber = Ask::Instrumentation.subscribe { |e| puts e.name }
Ask::Instrumentation.unsubscribe(subscriber)

Parameters:

  • subscriber_or_name (ActiveSupport::Notifications::Fanout::Subscriber, String, Regexp) —

    The subscriber to remove



94
95
96
 
# File 'lib/ask/instrumentation.rb', line 94

def unsubscribe(subscriber_or_name)
  ActiveSupport::Notifications.unsubscribe(subscriber_or_name)
end

.with_metadata(metadata = {}) { ... } ⇒ Object

Execute a block with thread-local metadata that gets merged into all events emitted within the block.

Nested calls merge the inner metadata into the outer, with the inner values taking precedence for duplicate keys. The outer metadata is restored after the inner block completes.

Metadata is useful for attaching context such as user_id, session_id, request_id, or trace_id to every event.

Examples:

Ask::Instrumentation.with_metadata(user_id: 42) do
  Ask::Instrumentation.instrument("chat.ask", provider: "openai", model: "gpt-4") do
    # ... event payload will include user_id: 42
  end
end

Parameters:

  • metadata (Hash) (defaults to: {}) —

    The metadata hash to attach

Yields:

  • The block to execute with the metadata context

Returns:

  • (Object) —

    The return value of the block



138
139
140
141
142
143
144
 
# File 'lib/ask/instrumentation.rb', line 138

def with_metadata(metadata = {})
  previous = Thread.current[:ask_instrumentation_metadata]
  Thread.current[:ask_instrumentation_metadata] = (previous || {}).merge(metadata)
  yield
ensure
  Thread.current[:ask_instrumentation_metadata] = previous
end