Module: RCrewAI::Multimodal

Defined in:
lib/rcrewai/multimodal.rb

Overview

Builds multimodal message content (text + images) in the OpenAI chat-completions format:

[{ type: 'text', text: '...' },
{ type: 'image_url', image_url: { url: '...' } }]

Local image paths are base64-encoded into data URLs; URLs pass through. Only OpenAI-style multimodal is supported today; other providers raise.

Defined Under Namespace

Classes: UnsupportedAttachmentError, UnsupportedProviderError

Constant Summary collapse

SUPPORTED_PROVIDERS = 
%i[openai azure].freeze
MIME_TYPES = 
{
  '.png' => 'image/png',
  '.jpg' => 'image/jpeg',
  '.jpeg' => 'image/jpeg',
  '.gif' => 'image/gif',
  '.webp' => 'image/webp'
}.freeze

Class Method Summary collapse

Class Method Details

.content_parts(text, attachments) ⇒ Object

Returns an OpenAI-style content-parts array for the given text and attachments. With no attachments this is a single text part.



28
29
30
31
32
 
# File 'lib/rcrewai/multimodal.rb', line 28

def content_parts(text, attachments)
  parts = [{ type: 'text', text: text.to_s }]
  Array(attachments).each { |att| parts << image_part(att) }
  parts
end

.data_url_for(path) ⇒ Object

Raises:



56
57
58
59
60
61
62
 
# File 'lib/rcrewai/multimodal.rb', line 56

def data_url_for(path)
  raise UnsupportedAttachmentError, 'image attachment needs a :url or :path' unless path

  mime = MIME_TYPES[File.extname(path).downcase] || 'application/octet-stream'
  encoded = Base64.strict_encode64(File.binread(path))
  "data:#{mime};base64,#{encoded}"
end

.ensure_supported_provider!(provider) ⇒ Object

Raises:



38
39
40
41
42
43
 
# File 'lib/rcrewai/multimodal.rb', line 38

def ensure_supported_provider!(provider)
  return if supported_provider?(provider)

  raise UnsupportedProviderError,
        "multimodal attachments are not supported for provider #{provider}"
end

.image_part(attachment) ⇒ Object

Raises:



45
46
47
48
49
50
51
52
53
54
 
# File 'lib/rcrewai/multimodal.rb', line 45

def image_part(attachment)
  type = attachment[:type] || attachment['type']
  raise UnsupportedAttachmentError, "unsupported attachment type: #{type.inspect}" unless type.to_sym == :image

  url = attachment[:url] || attachment['url']
  path = attachment[:path] || attachment['path']
  resolved = url || data_url_for(path)

  { type: 'image_url', image_url: { url: resolved } }
end

.supported_provider?(provider) ⇒ Boolean

Returns:

  • (Boolean) 


34
35
36
 
# File 'lib/rcrewai/multimodal.rb', line 34

def supported_provider?(provider)
  SUPPORTED_PROVIDERS.include?(provider.to_sym)
end