Module: Insika::Media

Defined in:
lib/insika/media.rb

Overview

WS9: the engine transports MEDIA, never meaning. Content parts ride the message contract — { "type": "text", "text": … }, { "type": "image", "url": … }, { "type": "audio", "url": … } — and the Executor turns them into a turn: audio is transcribed (text marked source: :voice), images attach to the model ask and the first URL is {{ctx.image_url}} for data/HTTP tools. This class owns the PURE parts (normalization) and the STT SEAM (injectable — specs stub it; the default fetches the audio and transcribes via RubyLLM behind a lazy require, so the core stays gem-free at load). Media::Output is the generated-media half (WS9, saída): the turn can PRODUCE an image or an audio clip when the agent opted in (AgentProfile#outputs) AND the channel declared it can receive it (channel.capabilities) — nothing leaks by default.

Defined Under Namespace

Modules: Output Classes: Part

Constant Summary collapse

OUTPUT_CAPABILITIES =

The OUTPUT media kinds a channel may declare it can receive (channel.capabilities). The closed list is the "abstraction admits only what leaks" rule: an unknown value is refused at the edge, never silently ignored.

%w[image_output audio_output].freeze
MAX_AUDIO_BYTES =

The ceilings on INBOUND media (a URL a consumer sent us). Both fetches stream into the cap and refuse past it: the bytes land in THIS process, so an uncapped one is a hostile URL away from growing it until it dies.

1_000_000
MAX_IMAGE_BYTES =

a voice note, not a warehouse

5_000_000

Class Method Summary collapse

Class Method Details

.audio_parts(parts) ⇒ Object

a photo, not a poster



83
 
# File 'lib/insika/media.rb', line 83

def self.audio_parts(parts) = parts.select(&:audio?)

.channel_capabilities(raw) ⇒ Object

-> [String]: the capabilities a raw channel hash declares. Lenient on the key spelling (symbol|string) at both boundaries (request parse vs persisted command payload); [] = the channel declared nothing.



72
73
74
75
 
# File 'lib/insika/media.rb', line 72

def self.channel_capabilities(raw)
  channel = raw.is_a?(Hash) ? raw : {}
  Array(channel[:capabilities] || channel["capabilities"]).map(&:to_s)
end

.default_transcriber(stt_model:, stt_language: nil) ⇒ Object

The STT seam: ->(url) { text } (default: fetch + RubyLLM transcription). Injected so a spec never touches the network; the default is built lazily when the turn first carries audio.



89
90
91
92
93
 
# File 'lib/insika/media.rb', line 89

def self.default_transcriber(stt_model:, stt_language: nil)
  lambda do |url|
    fetch_and_transcribe(url, model: stt_model, language: stt_language)
  end
end

.egress_opt_out ⇒ Object

The opt-out the comment above promises, read from the SAME env the data-tool guard reads (INSIKA_EGRESS_ALLOW_HTTP / _ALLOW_PRIVATE): without this, a local run serving media over http:// ALWAYS failed, however the deployment was configured. INSIKA_EGRESS_HOSTS is deliberately NOT applied: that allowlist pins the handful of hosts a tool may call, while media URLs come from the channel's CDN — honouring it here would break every real deployment that narrows its tools.



138
139
140
141
 
# File 'lib/insika/media.rb', line 138

def self.egress_opt_out
  { allow_http: Insika::EnvSchema.truthy?(ENV["INSIKA_EGRESS_ALLOW_HTTP"]),
    allow_private: Insika::EnvSchema.truthy?(ENV["INSIKA_EGRESS_ALLOW_PRIVATE"]) }
end

.fetch_and_transcribe(url, model:, language:) ⇒ Object 



95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/insika/media.rb', line 95

def self.fetch_and_transcribe(url, model:, language:)
  require "net/http"
  require "uri"
  require "ruby_llm" # lazy — the core loads without it (load-guard)

  bytes = fetch_binary(url)
  audio = RubyLLM::Attachment.new(bytes)
  options = { model: model, assume_model_exists: true }
  options[:language] = language if language
  RubyLLM::Transcription.transcribe(audio, **options).text
end

.fetch_binary(url, max_bytes: MAX_AUDIO_BYTES) ⇒ Object

Egress-guarded binary fetch of a media URL. Blocked like the webhook: the url is consumer config/input, so a private/loopback/metadata target is refused (SSRF) unless the deployment opts out. Size-capped (the caller picks the ceiling; the default is the audio one).



111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
 
# File 'lib/insika/media.rb', line 111

def self.fetch_binary(url, max_bytes: MAX_AUDIO_BYTES)
  violation = Insika::EgressGuard.violation(url, **egress_opt_out)
  raise Insika::MediaError, "media egress blocked for #{url}: #{violation}" if violation

  uri = URI.parse(url)
  opts = { use_ssl: uri.scheme == "https", open_timeout: 30, read_timeout: 60 }
  Net::HTTP.start(uri.host, uri.port, opts) do |http|
    buf = +"".b
    http.request(Net::HTTP::Get.new(uri)) do |resp|
      raise Insika::MediaError, "media fetch HTTP #{resp.code}" unless resp.is_a?(Net::HTTPSuccess)

      resp.read_body { |chunk| buf << chunk; break if buf.bytesize > max_bytes }
    end
    raise Insika::MediaError, "media exceeds #{max_bytes} bytes" if buf.bytesize > max_bytes
    buf
  end
rescue URI::InvalidURIError
  raise Insika::MediaError, "invalid media URL"
end

.image_parts(parts) ⇒ Object 



84
 
# File 'lib/insika/media.rb', line 84

def self.image_parts(parts) = parts.select(&:image?)

.parts(raw) ⇒ Object

-> [Part]: normalize the raw parts (string|symbol keys), skipping anything that is not a well-formed text/image/audio part. Lenient on purpose — the SURFACE validates the contract with well_formed? (a malformed part is a 422 before dispatch); here a stray entry must not break the turn.



28
29
30
31
32
33
34
35
36
37
38
39
40
41
 
# File 'lib/insika/media.rb', line 28

def self.parts(raw)
  Array(raw).filter_map do |p|
    next unless p.is_a?(Hash)

    type = (p[:type] || p["type"]).to_s
    url = (p[:url] || p["url"]).to_s
    text = (p[:text] || p["text"]).to_s
    case type
    when "text" then text.empty? ? nil : Part.new("text", text, nil)
    when "image", "audio" then url.empty? ? nil : Part.new(type, nil, url)
    else nil
    end
  end
end

.well_formed?(raw) ⇒ Boolean

The SURFACE's contract check (server edge): true when EVERY entry is a well-formed content part — a Hash whose type is text (with text), image or audio (with url). The edge raises a 422 on the first offender; the engine itself stays lenient (parts skips strays so a non-HTTP transport that bypassed the edge cannot break a turn).

Returns:

  • (Boolean) 


48
49
50
51
52
53
54
55
56
57
58
59
60
61
 
# File 'lib/insika/media.rb', line 48

def self.well_formed?(raw)
  Array(raw).all? do |p|
    next false unless p.is_a?(Hash)

    case (p[:type] || p["type"]).to_s
    when "text" then !(p[:text] || p["text"]).to_s.empty?
    when "image", "audio" then !(p[:url] || p["url"]).to_s.empty?
    # a part WITHOUT a type is admitted only as a bare text part (the
    # shape the input joiner already tolerates) — anything else is refused.
    when "" then !(p[:text] || p["text"]).to_s.empty?
    else false
    end
  end
end