Skip to navigation

v2.6.0 — Translate video, project editing, and video edits

Translate the speech in a video into several languages in one run with POST /v1/workflows/translate-video, or dub a single file with POST /v1/tools/dub-video. Each language becomes its own editable video, and export and video edits can target one video in a multi-video project with editableVideoId. This release also adds project editing: inspect a project’s videos and saved workflows, then apply typed edits with safe retries and undo. Remix actions are now called video edits: workflow starts take videoEdits, and POST /v1/projects/{projectId}/video-edits replaces the remix endpoints. MCP clients can delegate work to the VideoGen assistant and import video links directly.

Added

  • Translate video workflow: POST /v1/workflows/translate-video takes an uploaded videoFileId (up to 8 minutes) and outputLanguages (one or more language codes, such as es and fr). Repeated languages are translated once. Pass sourceLanguage when you know the spoken language; otherwise it is detected. The original speakers’ voices and timing are kept. Languages are built in parallel, and one failing language does not stop the others. The run fails only when every language fails. Optional fields are videoEdits (applied to every language’s video), autoExport (renders the first successful language), exportOptions, and isOutputTemporary. Credit use grows with the video’s length and the number of languages. Requires a Pro subscription.
  • Workflow run outputs: GET /v1/workflows/runs/{workflowRunId} now returns outputs, one entry per editable video the run produced, with its editableVideoId (vg_vide_...), language, and status (succeeded or failed). Translate-video runs list one entry per requested language.
  • Translate a video’s speech: POST /v1/tools/dub-video takes videoFileId and targetLanguage, plus an optional sourceLanguage, and returns one translated video file without creating a project. Billed per second of output video. Requires a Pro subscription.
  • Dub videos when translating a project: the TRANSLATE_PROJECT video edit accepts dubVideos (default false). When true, the speech in the project’s uploaded videos is also dubbed into the target language, billed per second of each video with speech. When false, those videos keep their original audio and their captions are still translated. Returns 400 if the target language does not support dubbing.
  • Video edits endpoints: POST /v1/projects/{projectId}/video-edits applies an ordered list of videoEdits to one video, named by the required editableVideoId. Set saveAsNewProject to edit that video in a copy. The response returns one videoEditId (vg_vedt_...) per edit. GET /v1/projects/{projectId}/video-edits lists their status and progress, optionally filtered by editableVideoId. See Video edits.
  • Project artifacts: GET /v1/projects/{projectId}/artifacts lists a project’s artifacts with their artifactId (vg_arti_...), type, name, revision, and the matching editableVideoId or workflowId. GET /v1/projects/{projectId}/artifacts/{artifactId} reads one artifact’s saved workflow settings or video identity. Page long text with textOffset / textLimit (UTF-16 code units) and storyboard scenes or slides with itemOffset / itemLimit. Reading never renders or generates anything.
  • Editing capabilities: GET /v1/projects/{projectId}/editing-capabilities returns whether you can edit the project (canEdit), every supported command type with its target and execution, and request limits (maxCommands, maxBodyBytes).
  • Inspect a video: GET /v1/projects/{projectId}/videos/{editableVideoId}/inspection pages through a video’s timeline. With no selector it returns section summaries. Pass sectionId for a section’s layers, layerId for a layer’s assets, or assetId for an asset’s transcript words and source timings. Page with offset / limit, and pass the returned revision back as expectedRevision so a change between pages returns a conflict instead of mixing versions.
  • Edit a video: POST /v1/projects/{projectId}/videos/{editableVideoId}/edits applies up to 100 typed commands in one atomic batch (mode: "COMMANDS", expectedRevision, mutationId, commands). Commands cover text, captions, sections, layers, asset position, trim, playback, filters, transitions, voiceover text, and background music. These edits do not generate media or export.
  • Background video edits: the same endpoint and the MCP edit_video tool also accept background commands on one video: SET_LOGO, REMOVE_LOGO, APPLY_TRANSITIONS, ADD_ZOOM, CLEAN_UP_TRANSCRIPT, CONVERT_IMAGES_TO_VIDEOS, REGENERATE_IMAGES, UPSCALE_ASSETS, CHANGE_NARRATOR, SHUFFLE_STOCK_VISUALS, GENERATE_MUSIC, TRANSLATE_PROJECT, ADD_MEDIA, ADD_DRAFT_MEDIA, GENERATE_DRAFT_MEDIA, and RUN_TOOL_ON_ASSET. Send a background command as the only command in its request, using sectionId and assetId values from the video’s inspection. The receipt returns outcome: "STARTED" and a videoEditId. Follow it with GET /v1/projects/{projectId}/video-edits. Retrying the same request returns the same videoEditId and does not start a second task. Generation commands spend credits. Editing capabilities list these commands with execution: "BACKGROUND".
  • Edit project documents: POST /v1/projects/{projectId}/mutations atomically changes the project title and description, artifact names, and saved workflow settings (script, storyboard scenes, slideshow slides, captions, aspect ratio, quality, voice speed, and background music). List the inspected revision of each changed document in expectedDocuments. Workflow settings changes do not regenerate or alter videos that were already generated.
  • Safe retries and undo: every edit takes a new caller-generated UUID mutationId and returns a receipt with outcome (APPLIED, NO_OP, CONFLICT, or STARTED) and projectRevision. A stale revision conflicts instead of overwriting newer changes. Retry the same request unchanged to get the original receipt, or read it with GET /v1/projects/{projectId}/mutations/{mutationId}. POST /v1/projects/{projectId}/mutations/{mutationId}/undo applies the inverse as a new edit (send a fresh mutationId in the body) and keeps other edits made since. Undo the resulting edit to redo. Undo never repeats generation or billing.
  • Access to projects created in the app: API keys and OAuth integrations can read and edit app-created and API-created projects that their authenticated caller can access on the selected team. Existing team and project permissions apply. Deleting an API key revokes its access.
  • Choose a video from a multi-video link: POST /v1/files/import-url accepts an optional videoIndex (1 to 20). When a link contains more than one video and no videoIndex was sent, GET /v1/files/imports/{fileImportId} ends as failed with a selection object listing each video’s videoIndex, title, and durationSeconds. No video is downloaded. Ask the user which video they want, then start a new import with the same sourceUrl and their videoIndex. Do not pick the first video on the user’s behalf.
  • SDKs and CLI:
    • TypeScript workflows.translateVideo / translateVideoAndWait, tools.dubVideo / dubVideoAndWait, and projects.startVideoEdits / startVideoEditsAndWait / listVideoEdits.
    • TypeScript project editing: projects.listProjectArtifacts, getProjectArtifact, getProjectEditingCapabilities, inspectEditableVideo, editVideo, applyProjectEdits, getProjectEdit, and undoProjectEdit (pass the undo’s own id as newMutationId).
    • Python has the snake_case equivalents (translate_video, dub_video, start_video_edits, list_video_edits, undo_project_edit with new_mutation_id, and so on).
    • The CLI adds videogen workflows translate-video, videogen tools dub-video, videogen projects start-video-edits and list-video-edits, and matching project editing commands such as edit-video and undo-project-edit.
  • MCP:
    • translate_video and dub_video. export_project accepts editableVideoId.
    • start_video_edits and list_video_edits replace remix_project and list_project_remix_actions.
    • Project editing: list_project_artifacts, get_project_artifact, get_project_editing_capabilities, inspect_video, edit_video, edit_project, edit_workflow, get_project_edit, and undo_project_edit.
    • Assistant: start_assistant_chat hands a new project request to the VideoGen assistant, and send_assistant_message continues work on an existing API-created project. get_assistant reads the conversation, get_assistant_message polls a reply, and act_on_assistant_action approves or rejects an action the assistant proposed.
    • Video link imports: import_video_from_url starts a link import and get_video_url_import polls it. A successful import returns a fileId to pass to video_to_video.
    • Media previews: on the hosted MCP server, show_media_preview displays up to 12 finished images, videos, or audio files in the conversation from their vg_file_... ids. It does not generate or change anything.
  • Agent frameworks: translate_video and dub_video tools in the OpenClaw plugin and the OpenAI Agents package.

Changed

  • Remix actions are now video edits: workflow start requests take videoEdits instead of remixActions, and workflow start responses return videoEditIds (vg_vedt_...) instead of remixActionIds. The edit types and their fields are unchanged. Rename the field in your requests and read videoEditIds from responses when you upgrade. The SDK types use the new names.
  • Export one video: POST /v1/projects/{projectId}/export accepts an optional editableVideoId from a run’s outputs.
  • Project status reports stopped generations: status on GET /v1/projects/{projectId} and GET /v1/projects can now be failed or cancelled when an unfinished project’s latest generation stopped. generating means work is still pending or running. Handle the two new values wherever you switch on status.
  • Assistant action labels: on assistant messages, actions[].label is the approval card’s title (default Apply edit) and actions[].detail.summary is its description.

Deprecated

  • Remix endpoints: POST /v1/projects/{projectId}/remix and GET /v1/projects/{projectId}/remix-actions, the SDK methods projects.remixProject, remixAndWait, and listProjectRemixActions (Python remix_project, remix_and_wait, list_project_remix_actions), the CLI remix-project and list-project-remix-actions commands, and the MCP remix_project and list_project_remix_actions tools still work but are no longer documented or listed. Move to POST /v1/projects/{projectId}/video-edits and GET /v1/projects/{projectId}/video-edits, which require an editableVideoId.

Fixed

  • Wrong file type fails up front: tool requests that pass a file of the wrong media type, such as a video id in startFrameFileId or imageFileIds, now return invalid_parameters immediately with the parameter, the expected type, and the file’s actual type. Previously the task started and then failed.
  • Short reference audio: generate-video-clip with a reference audio file shorter than 2 seconds now fails quickly with a message naming the file’s length, without retries or charges.