Class: Clacky::Media::Generator

Inherits:
Object
  • Object
show all
Defined in:
lib/clacky/media/generator.rb

Overview

Top-level dispatcher: takes an AgentConfig and a request, picks the right provider class based on the configured image model's base_url, and delegates.

Adding a new modality (video / audio) means:

1. add a generate_<modality> method here that resolves the correct
 type=<modality> entry and class
2. add a provider class under lib/clacky/media/ implementing the call

Constant Summary collapse

GOOGLE_NATIVE_HOSTS =

Hosts that speak the native Google AI Studio API instead of an OpenAI-compatible facade. Matched as a substring against the configured base_url so any regional / staging variant is caught.

[
  "generativelanguage.googleapis.com",
  "aiplatform.googleapis.com"
].freeze
DASHSCOPE_NATIVE_HOSTS =

Hosts that speak Alibaba's native DashScope (Qwen-Image) API instead of an OpenAI-compatible facade. Matched as a substring so every regional variant (dashscope / dashscope-intl / dashscope-us, and the Singapore *.maas.aliyuncs.com workspace hosts) is caught. Third-party aggregators (SiliconFlow, OpenRouter, …) that re-expose qwen-image behind an OpenAI-compatible endpoint are NOT under aliyuncs.com, so they correctly keep going through OpenAICompat.

[
  "aliyuncs.com"
].freeze
VOLCENGINE_NATIVE_HOSTS =

Hosts that speak Volcengine Ark's native async video-task API (ByteDance Doubao Seedance) instead of an OpenAI-compatible facade. Matched as a substring so regional variants are caught.

[
  "volces.com"
].freeze

Instance Method Summary collapse

Constructor Details

#initialize(agent_config) ⇒ Generator

Returns a new instance of Generator.

Parameters:



46
47
48
 
# File 'lib/clacky/media/generator.rb', line 46

def initialize(agent_config)
  @agent_config = agent_config
end

Instance Method Details

#audio_model_entry ⇒ Hash?

Returns the type=audio model entry, or nil if not configured.

Returns:

  • (Hash, nil) —

    the type=audio model entry, or nil if not configured



61
62
63
 
# File 'lib/clacky/media/generator.rb', line 61

def audio_model_entry
  @agent_config.find_model_by_type("audio")
end

#generate_image(prompt:, aspect_ratio: "landscape", output_dir: nil, **kwargs) ⇒ Object 



65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/clacky/media/generator.rb', line 65

def generate_image(prompt:, aspect_ratio: "landscape", output_dir: nil, **kwargs)
  entry = image_model_entry
  if entry.nil?
    return {
      "success"    => false,
      "image"      => nil,
      "error"      => "No image model configured. Add a model with type=image in settings.",
      "error_type" => "not_configured",
      "provider"   => "",
      "model"      => "",
      "prompt"     => prompt
    }
  end

  provider = build_provider_for(entry)
  provider.generate_image(
    prompt: prompt,
    aspect_ratio: aspect_ratio,
    output_dir: output_dir,
    **kwargs
  )
end

#generate_speech(input:, voice: nil, output_dir: nil, **kwargs) ⇒ Object 



140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/clacky/media/generator.rb', line 140

def generate_speech(input:, voice: nil, output_dir: nil, **kwargs)
  entry = audio_model_entry
  if entry.nil?
    return {
      "success"    => false,
      "audio"      => nil,
      "error"      => "No audio model configured. Add a model with type=audio in settings.",
      "error_type" => "not_configured",
      "provider"   => "",
      "model"      => "",
      "input"      => input
    }
  end

  provider = build_provider_for(entry)
  provider.generate_speech(
    input: input,
    voice: voice,
    output_dir: output_dir,
    **kwargs
  )
end

#generate_transcription(audio_base64:, mime_type:, **kwargs) ⇒ Object 



171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
 
# File 'lib/clacky/media/generator.rb', line 171

def generate_transcription(audio_base64:, mime_type:, **kwargs)
  entry = stt_model_entry
  if entry.nil?
    return {
      "success"    => false,
      "text"       => nil,
      "error"      => "No STT model configured. Add a model with type=stt in settings.",
      "error_type" => "not_configured",
      "provider"   => "",
      "model"      => ""
    }
  end

  provider = build_provider_for(entry)
  provider.generate_transcription(
    audio_base64: audio_base64,
    mime_type: mime_type,
    **kwargs
  )
end

#generate_video(prompt:, aspect_ratio: "landscape", duration_seconds: nil, output_dir: nil, **kwargs) ⇒ Object 



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/clacky/media/generator.rb', line 88

def generate_video(prompt:, aspect_ratio: "landscape", duration_seconds: nil, output_dir: nil, **kwargs)
  entry = video_model_entry
  if entry.nil?
    return {
      "success"    => false,
      "video"      => nil,
      "error"      => "No video model configured. Add a model with type=video in settings.",
      "error_type" => "not_configured",
      "provider"   => "",
      "model"      => "",
      "prompt"     => prompt
    }
  end

  provider = build_provider_for(entry)
  provider.generate_video(
    prompt: prompt,
    aspect_ratio: aspect_ratio,
    duration_seconds: duration_seconds,
    output_dir: output_dir,
    **kwargs
  )
end

#image_model_entry ⇒ Hash?

Returns the type=image model entry, or nil if not configured.

Returns:

  • (Hash, nil) —

    the type=image model entry, or nil if not configured



51
52
53
 
# File 'lib/clacky/media/generator.rb', line 51

def image_model_entry
  @agent_config.find_model_by_type("image")
end

#stt_model_entry ⇒ Object 



163
164
165
 
# File 'lib/clacky/media/generator.rb', line 163

def stt_model_entry
  @agent_config.find_model_by_type("stt")
end

#understand_video(video_base64:, mime_type:, prompt: nil, **kwargs) ⇒ Object 



192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
 
# File 'lib/clacky/media/generator.rb', line 192

def understand_video(video_base64:, mime_type:, prompt: nil, **kwargs)
  entry = video_understanding_model_entry
  if entry.nil?
    return {
      "success"    => false,
      "analysis"   => nil,
      "error"      => "No video understanding model configured. Add a model with type=video_understanding in settings.",
      "error_type" => "not_configured",
      "provider"   => "",
      "model"      => "",
      "prompt"     => prompt
    }
  end

  provider = build_provider_for(entry)
  provider.understand_video(
    video_base64: video_base64,
    mime_type: mime_type,
    prompt: prompt,
    **kwargs
  )
end

#video_model_entry ⇒ Hash?

Returns the type=video model entry, or nil if not configured.

Returns:

  • (Hash, nil) —

    the type=video model entry, or nil if not configured



56
57
58
 
# File 'lib/clacky/media/generator.rb', line 56

def video_model_entry
  @agent_config.find_model_by_type("video")
end

#video_status(task_id:, output_dir: nil) ⇒ Object 



112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
 
# File 'lib/clacky/media/generator.rb', line 112

def video_status(task_id:, output_dir: nil)
  entry = video_model_entry
  if entry.nil?
    return {
      "success"    => false,
      "status"     => "failed",
      "error"      => "No video model configured. Add a model with type=video in settings.",
      "error_type" => "not_configured",
      "provider"   => "",
      "model"      => ""
    }
  end

  provider = build_provider_for(entry)
  unless provider.respond_to?(:video_status)
    return {
      "success"    => false,
      "status"     => "failed",
      "error"      => "Status queries are only supported for Volcengine (Seedance) video; this provider generates synchronously.",
      "error_type" => "unsupported",
      "provider"   => "",
      "model"      => ""
    }
  end

  provider.video_status(task_id: task_id, output_dir: output_dir)
end

#video_understanding_model_entry ⇒ Object 



167
168
169
 
# File 'lib/clacky/media/generator.rb', line 167

def video_understanding_model_entry
  @agent_config.find_model_by_type("video_understanding")
end