Class: Clacky::Media::Volcengine

Inherits:
Base
  • Object
show all
Defined in:
lib/clacky/media/volcengine.rb

Overview

Volcengine Ark (ByteDance Doubao Seedance) video generation provider.

Ark is NOT OpenAI-compatible for video. It uses an asynchronous task model with its own request envelope: submit a task to POST /api/v3/contents/generations/tasks, then poll GET /api/v3/contents/generations/tasks/id until it succeeds, fails, or expires. Unlike the blocking Veo path, we surface that async model end to end: #generate_video submits and returns the task id immediately, and #video_status polls a single time (downloading the MP4 once the task has succeeded). This keeps a slow (e.g. 4k) render from blocking the HTTP request past its timeout and dropping the task id.

Routing: Generator sends any base_url under *.volces.com here. We derive the API root from the host so users can paste the standard base_url they use for Ark chat models (…/api/v3) and still get working video generation.

Seedance content is multimodal: alongside the text prompt, callers may attach a first frame, a last frame, reference images (0-9), reference videos (0-3) and reference audios (0-3). Each media item may be a public http(s) URL, a data URL, a local file path (encoded to a data URL), or a { "b64_json" => ..., "mime_type" => ... } hash.

Constant Summary collapse

TASKS_PATH = 
"/api/v3/contents/generations/tasks"
PROVIDER_ID = 
"volcengine"
ASPECT_TO_RATIO =

aspect_ratio -> Ark ratio. Ark also accepts more granular ratios and "adaptive"; when the caller passes one of those verbatim we forward it unchanged (see #resolve_ratio).

{
  "landscape" => "16:9",
  "portrait"  => "9:16",
  "square"    => "1:1"
}.freeze
ARK_RATIOS =

Ratios Ark accepts directly. Anything else falls back to the aspect_ratio mapping / default.

%w[16:9 4:3 1:1 3:4 9:16 21:9 adaptive].freeze
DEFAULT_RATIO = 
"16:9"
DEFAULT_RESOLUTION =

Explicit cost-control default: when the caller omits resolution we pin 720p rather than relying on Ark's server-side default (which could change and silently raise the user's bill).

"720p"

Instance Method Summary collapse

Methods inherited from Base

#generate_image, #generate_speech, #generate_transcription, #initialize, #understand_video

Constructor Details

This class inherits a constructor from Clacky::Media::Base

Instance Method Details

#generate_video(prompt:, aspect_ratio: "landscape", duration_seconds: nil, output_dir: nil, image: nil, first_frame: nil, last_frame: nil, reference_images: nil, reference_videos: nil, reference_audios: nil, resolution: nil, generate_audio: nil, watermark: nil, seed: nil, **_kwargs) ⇒ Object

Parameters:

  • prompt (String) —

    the video prompt (may be empty when driven purely by reference media, but Ark still recommends text)

  • aspect_ratio (String) (defaults to: "landscape") —

    "landscape"/"portrait"/"square", or an Ark ratio string ("16:9", "9:16", "adaptive", ...)

  • duration_seconds (Integer, nil) (defaults to: nil) —

    target duration; -1 lets the model choose (Seedance 2.0 / 1.5 Pro only)

  • first_frame (String, Hash, nil) (defaults to: nil) —

    first frame image

  • last_frame (String, Hash, nil) (defaults to: nil) —

    last frame image

  • reference_images (Array, String, Hash, nil) (defaults to: nil) —

    reference images

  • reference_videos (Array, String, Hash, nil) (defaults to: nil) —

    reference videos

  • reference_audios (Array, String, Hash, nil) (defaults to: nil) —

    reference audios

  • resolution (String, nil) (defaults to: nil) —

    "480p"/"720p"/"1080p"/"4k"

  • generate_audio (Boolean, nil) (defaults to: nil) —

    whether to synthesize audio

  • watermark (Boolean, nil) (defaults to: nil) —

    whether to add a watermark

  • seed (Integer, nil) (defaults to: nil) —

    random seed



72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
 
# File 'lib/clacky/media/volcengine.rb', line 72

def generate_video(prompt:, aspect_ratio: "landscape", duration_seconds: nil, output_dir: nil,
                   image: nil, first_frame: nil, last_frame: nil,
                   reference_images: nil, reference_videos: nil, reference_audios: nil,
                   resolution: nil, generate_audio: nil, watermark: nil, seed: nil, **_kwargs)
  # `image` is the legacy single first-frame field used across the media
  # stack; treat it as first_frame when no explicit first_frame is given.
  first_frame ||= image

  content, build_err = build_content(
    prompt: prompt,
    first_frame: first_frame,
    last_frame: last_frame,
    reference_images: reference_images,
    reference_videos: reference_videos,
    reference_audios: reference_audios
  )
  if build_err
    return video_error_response(
      error: build_err, error_type: "invalid_argument",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  has_text  = prompt.to_s.strip != ""
  has_media = content.length > (has_text ? 1 : 0)
  if !has_text && !has_media
    return video_error_response(
      error: "A text prompt or at least one reference image/video is required",
      error_type: "invalid_argument",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  if @api_key.to_s.empty?
    return video_error_response(
      error: "api_key not configured for video model '#{@model}'",
      error_type: "auth_required",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  ratio   = resolve_ratio(aspect_ratio)
  payload = { model: @model, content: content }
  payload[:ratio]          = ratio unless ratio.nil?
  payload[:duration]       = duration_seconds.to_i if duration_seconds
  payload[:resolution]     = (resolution && !resolution.to_s.empty?) ? resolution.to_s : DEFAULT_RESOLUTION
  payload[:generate_audio] = generate_audio unless generate_audio.nil?
  payload[:watermark]      = watermark unless watermark.nil?
  payload[:seed]           = seed.to_i if seed

  begin
    response = connection.post(TASKS_PATH) do |req|
      req.headers["Content-Type"]  = "application/json"
      req.headers["Authorization"] = "Bearer #{@api_key}"
      req.body = JSON.generate(payload)
    end
  rescue Faraday::Error => e
    return video_error_response(
      error: "HTTP request failed: #{e.message}",
      error_type: "network_error",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  body = parse_json(response.body)
  unless body.is_a?(Hash)
    return video_error_response(
      error: "Invalid JSON response from upstream",
      error_type: "invalid_response",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  # Ark reports submission errors via a nested "error" object.
  if body["error"].is_a?(Hash)
    err = body["error"]
    return video_error_response(
      error: "Upstream error #{err["code"]}: #{err["message"]}",
      error_type: "api_error",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  unless response.success?
    return video_error_response(
      error: "Upstream #{response.status}: #{truncate(response.body, 500)}",
      error_type: "api_error",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  task_id = body["id"]
  if task_id.nil? || task_id.to_s.empty?
    return video_error_response(
      error: "Upstream did not return a task id",
      error_type: "empty_response",
      provider: PROVIDER_ID, prompt: prompt, aspect_ratio: aspect_ratio
    )
  end

  # Submit only: return the task id immediately. The caller polls
  # GET /api/media/video/status so a long (e.g. 4k) render never blocks
  # the HTTP request past its timeout — which used to drop the task id
  # and push the model into resubmitting, doubling the bill.
  {
    "success"      => true,
    "status"       => "submitted",
    "task_id"      => task_id,
    "provider"     => PROVIDER_ID,
    "model"        => @model,
    "prompt"       => prompt,
    "aspect_ratio" => aspect_ratio,
    "ratio"        => ratio,
    "duration_seconds" => (duration_seconds ? duration_seconds.to_i : nil)
  }.compact
end

#video_status(task_id:, output_dir: nil) ⇒ Object

Poll a previously submitted task once. On "succeeded" the MP4 is downloaded and a local path is returned; other states map to running / failed without blocking.



192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
 
# File 'lib/clacky/media/volcengine.rb', line 192

def video_status(task_id:, output_dir: nil)
  if task_id.to_s.strip.empty?
    return video_error_response(
      error: "task_id is required", error_type: "invalid_argument",
      provider: PROVIDER_ID
    )
  end

  state, detail = fetch_task(task_id)
  case state
  when :running
    { "success" => true, "status" => "running", "task_id" => task_id, "provider" => PROVIDER_ID, "model" => @model }
  when :succeeded
    local_path = save_image_from_url(detail, output_dir: output_dir || Dir.pwd, prefix: "vid", extension: "mp4")
    if local_path.nil?
      return {
        "success" => false, "status" => "failed", "task_id" => task_id, "provider" => PROVIDER_ID,
        "model" => @model, "error" => "Failed to download generated video from #{detail}",
        "error_type" => "download_failed"
      }
    end
    {
      "success" => true, "status" => "succeeded", "video" => local_path,
      "task_id" => task_id, "provider" => PROVIDER_ID, "model" => @model
    }
  else # :failed
    {
      "success" => false, "status" => "failed", "task_id" => task_id, "provider" => PROVIDER_ID,
      "model" => @model, "error" => detail, "error_type" => "task_failed"
    }
  end
end