Module: Insika::Media

Defined in:
lib/insika/media.rb

Overview

WS9: the engine transports MEDIA, never meaning. Content parts ride the message contract — { "type": "text", "text": … }, { "type": "image", "url": … }, { "type": "audio", "url": … }, { "type": "document", "url": … } — and the Executor turns them into a turn: audio is transcribed (text marked source: :voice), images and documents attach to the model ask and the first URL of each kind is {{ctx.image_url}} / {{ctx.document_url}} for data tools. This class owns the PURE parts (normalization) and the STT SEAM (injectable — specs stub it; the default fetches the audio and transcribes via RubyLLM behind a lazy require, so the core stays gem-free at load). Media::Output is the generated-media half (WS9, saída): the turn can PRODUCE an image or an audio clip when the agent opted in (AgentProfile#outputs) AND the channel declared it can receive it (channel.capabilities) — nothing leaks by default.

Defined Under Namespace

Modules: Output Classes: Part

Constant Summary collapse

OUTPUT_CAPABILITIES =

The OUTPUT media kinds a channel may declare it can receive (channel.capabilities). The closed list is the "abstraction admits only what leaks" rule: an unknown value is refused at the edge, never silently ignored.

%w[image_output audio_output].freeze
MAX_AUDIO_BYTES =

The ceilings on INBOUND media (a URL a consumer sent us). Both fetches stream into the cap and refuse past it: the bytes land in THIS process, so an uncapped one is a hostile URL away from growing it until it dies.

1_000_000
MAX_IMAGE_BYTES =

a voice note, not a warehouse

5_000_000
MAX_DOCUMENT_BYTES =

a photo, not a poster

10_000_000

Class Method Summary collapse

Class Method Details

.audio_parts(parts) ⇒ Object

a prescription, not an archive



87
 
# File 'lib/insika/media.rb', line 87

def self.audio_parts(parts) = parts.select(&:audio?)

.channel_capabilities(raw) ⇒ Object

-> [String]: the capabilities a raw channel hash declares. Lenient on the key spelling (symbol|string) at both boundaries (request parse vs persisted command payload); [] = the channel declared nothing.



75
76
77
78
 
# File 'lib/insika/media.rb', line 75

def self.channel_capabilities(raw)
  channel = raw.is_a?(Hash) ? raw : {}
  Array(channel[:capabilities] || channel["capabilities"]).map(&:to_s)
end

.default_transcriber(stt_model:, stt_language: nil, stt_prompt: nil) ⇒ Object

The STT seam: ->(url) { text } (default: fetch + RubyLLM transcription). Injected so a spec never touches the network; the default is built lazily when the turn first carries audio. stt_prompt is the Whisper-family vocabulary hint (product names, brand terms) — OPERATOR config (agent profile / deployment env), never customer input.



96
97
98
99
100
 
# File 'lib/insika/media.rb', line 96

def self.default_transcriber(stt_model:, stt_language: nil, stt_prompt: nil)
  lambda do |url|
    fetch_and_transcribe(url, model: stt_model, language: stt_language, prompt: stt_prompt)
  end
end

.document_parts(parts) ⇒ Object 



89
 
# File 'lib/insika/media.rb', line 89

def self.document_parts(parts) = parts.select(&:document?)

.egress_opt_out ⇒ Object

The opt-out the comment above promises, read from the SAME env the data-tool guard reads (INSIKA_EGRESS_ALLOW_HTTP / _ALLOW_PRIVATE): without this, a local run serving media over http:// ALWAYS failed, however the deployment was configured. INSIKA_EGRESS_HOSTS is deliberately NOT applied: that allowlist pins the handful of hosts a tool may call, while media URLs come from the channel's CDN — honouring it here would break every real deployment that narrows its tools.



187
188
189
190
 
# File 'lib/insika/media.rb', line 187

def self.egress_opt_out
  { allow_http: Insika::EnvSchema.truthy?(ENV["INSIKA_EGRESS_ALLOW_HTTP"]),
    allow_private: Insika::EnvSchema.truthy?(ENV["INSIKA_EGRESS_ALLOW_PRIVATE"]) }
end

.fetch_and_transcribe(url, model:, language:, prompt: nil) ⇒ Object

A file PATH, not bytes and not an Attachment: RubyLLM::Transcription. transcribe hands its argument to the PROVIDER's own transcribe, and the two shapes in this gem disagree — Gemini wraps it in Attachment.new itself (a raw byte String there is misread as a Pathname and blows up on any embedded null byte, which real audio has), while the DEFAULT Provider#transcribe (OpenAI, Mistral) calls File.expand_path on it directly and cannot take bytes/IO/Attachment at all. A tempfile is the one shape both accept. assume_model_exists is deliberately NOT passed: RubyLLM raises ArgumentError when it's true without an explicit provider (see ModelSelection#assume_model_exists?), and stt_model here is a bare ref like utility_model elsewhere — the registry resolves it.



113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
 
# File 'lib/insika/media.rb', line 113

def self.fetch_and_transcribe(url, model:, language:, prompt: nil)
  require "net/http"
  require "uri"
  require "ruby_llm" # lazy — the core loads without it (load-guard)
  require "tempfile"

  bytes = fetch_binary(url)
  options = { model: model }
  options[:language] = language if language
  options[:prompt] = prompt if prompt
  Tempfile.create(["insika-media-", File.extname(filename_for(url).to_s)]) do |file|
    file.binmode
    file.write(bytes)
    file.flush
    RubyLLM::Transcription.transcribe(file.path, **options).text
  end
end

.fetch_binary(url, max_bytes: MAX_AUDIO_BYTES) ⇒ Object

Egress-guarded binary fetch of a media URL. Blocked like the webhook: the url is consumer config/input, so a private/loopback/metadata target is refused (SSRF) unless the deployment opts out. Size-capped (the caller picks the ceiling; the default is the audio one).



160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
 
# File 'lib/insika/media.rb', line 160

def self.fetch_binary(url, max_bytes: MAX_AUDIO_BYTES)
  violation = Insika::EgressGuard.violation(url, **egress_opt_out)
  raise Insika::MediaError, "media egress blocked for #{url}: #{violation}" if violation

  uri = URI.parse(url)
  opts = { use_ssl: uri.scheme == "https", open_timeout: 30, read_timeout: 60 }
  Net::HTTP.start(uri.host, uri.port, opts) do |http|
    buf = +"".b
    http.request(Net::HTTP::Get.new(uri)) do |resp|
      raise Insika::MediaError, "media fetch HTTP #{resp.code}" unless resp.is_a?(Net::HTTPSuccess)

      resp.read_body { |chunk| buf << chunk; break if buf.bytesize > max_bytes }
    end
    raise Insika::MediaError, "media exceeds #{max_bytes} bytes" if buf.bytesize > max_bytes
    buf
  end
rescue URI::InvalidURIError
  raise Insika::MediaError, "invalid media URL"
end

.filename_for(url) ⇒ Object

The URL's basename, for the attachment's mime sniff (".png" -> image/png; a URL with no filename falls back to the content sniff RubyLLM does).



148
149
150
151
152
153
154
 
# File 'lib/insika/media.rb', line 148

def self.filename_for(url)
  require "uri"
  name = File.basename(URI.parse(url).path.to_s)
  name.empty? ? nil : name
rescue URI::InvalidURIError
  nil
end

.image_parts(parts) ⇒ Object 



88
 
# File 'lib/insika/media.rb', line 88

def self.image_parts(parts) = parts.select(&:image?)

.parts(raw) ⇒ Object

-> [Part]: normalize the raw parts (string|symbol keys), skipping anything that is not a well-formed text/image/audio/document part. Lenient on purpose — the SURFACE validates the contract with well_formed? (a malformed part is a 422 before dispatch); here a stray entry must not break the turn.



31
32
33
34
35
36
37
38
39
40
41
42
43
44
 
# File 'lib/insika/media.rb', line 31

def self.parts(raw)
  Array(raw).filter_map do |p|
    next unless p.is_a?(Hash)

    type = (p[:type] || p["type"]).to_s
    url = (p[:url] || p["url"]).to_s
    text = (p[:text] || p["text"]).to_s
    case type
    when "text" then text.empty? ? nil : Part.new("text", text, nil)
    when "image", "audio", "document" then url.empty? ? nil : Part.new(type, nil, url)
    else nil
    end
  end
end

.url_attachment(url, max_bytes: MAX_IMAGE_BYTES) ⇒ Object

An inbound URL -> a RubyLLM::Attachment over bytes WE fetched (egress- guarded, size-capped — the media_attachment recipe). Shared by the Executor (inbound image/document parts) and Output.generate_image (edit sources / mask): an io-like source (StringIO) is the branch of Attachment that takes bytes already held, so the provider gets base64 rather than the URL — handing the raw URL to RubyLLM instead would leave the (uncapped) fetch to the gem.



138
139
140
141
142
143
144
 
# File 'lib/insika/media.rb', line 138

def self.url_attachment(url, max_bytes: MAX_IMAGE_BYTES)
  require "ruby_llm"
  require "stringio"

  bytes = fetch_binary(url, max_bytes: max_bytes)
  RubyLLM::Attachment.new(StringIO.new(bytes), filename: filename_for(url))
end

.well_formed?(raw) ⇒ Boolean

The SURFACE's contract check (server edge): true when EVERY entry is a well-formed content part — a Hash whose type is text (with text), image, audio or document (with url). The edge raises a 422 on the first offender; the engine itself stays lenient (parts skips strays so a non-HTTP transport that bypassed the edge cannot break a turn).

Returns:

  • (Boolean) 


51
52
53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/insika/media.rb', line 51

def self.well_formed?(raw)
  Array(raw).all? do |p|
    next false unless p.is_a?(Hash)

    case (p[:type] || p["type"]).to_s
    when "text" then !(p[:text] || p["text"]).to_s.empty?
    when "image", "audio", "document" then !(p[:url] || p["url"]).to_s.empty?
    # a part WITHOUT a type is admitted only as a bare text part (the
    # shape the input joiner already tolerates) — anything else is refused.
    when "" then !(p[:text] || p["text"]).to_s.empty?
    else false
    end
  end
end