Omni API
Omni API Docs

Models

Model groups, task types, and public readiness rules.

Models

The public catalog only lists model groups that are ready for real traffic. Model groups can share the same worker platform while keeping independent queue, scaling, pricing, and task schema rules.

Categories

Category Current status
Image Omni Image AIO is visible and ready.
Video Omni Video AIO is visible and ready.
Multi Omni Multi AIO is visible and ready for mixed media tasks.
Audio MM Audio AIO is visible and ready for sound generation.
VLLM Standalone VLLM groups remain hidden until model selection and text protocol are ready.

Omni Image AIO

Image tasks cover single image editing, local inpaint through the website wrapper, and multi-image-compatible API shape for future models. The public API exposes supported task types only; website-only wrappers can stay hidden from API users.

Qwen image tasks use the Catalog dimension_policy as the only product-size mapping. The current Fast, Eco, and periodic TargetSize is 1024; 896 belongs to the guest-lifetime intent. Public API requests use the Fast budget only. Standard text-to-image and image-edit requests submit matching reference_width/reference_height equal to the selected intent budget. Text-to-image also submits one of the five fixed aspect_ratio values; the Worker derives the 16-aligned canvas from the ratio and TargetSize. API callers may instead submit a smaller explicit 16-aligned text-to-image canvas with an empty aspect_ratio, provided its area stays within the Fast budget. Single-image edit omits aspect_ratio; dual-image edit supports auto and the five fixed ratios. Public requests never submit ComfyUI width, height, or target_size. Source images over 1536 pixels on the longest side are proportionally reduced by the Worker.

Omni Video AIO

Video tasks cover text-to-video, image-to-video, first-last-frame video, and video-to-video. Video extend is treated as video-to-video naming compatibility.

Every video task must submit matching reference_width and reference_height values from 384, 512, or 800. Media-based auto sizing uses the source aspect ratio and falls back to 1:1 when dimensions cannot be read.

Omni Multi AIO

Omni Multi AIO runs mixed media tasks on a shared runtime that maintains multiple model groups. The current public catalog exposes:

  • image_upscale for image upscaling.
  • audio_transcribe for standalone audio transcription.
  • voice_clone_tts for speech synthesis from text and reference audio.
  • vlm_chat for image-aware text responses.
  • talk2prompt for audio transcription followed by VLM prompt generation.
  • voice_translate for transcription, translation, and synthesized target-language audio.

All six ready task types use the same public Catalog and create contract. The live catalog remains authoritative.

Text and JSON outputs are stored directly in the task result. Image outputs use the normal signed R2 output contract. The runtime executes one task per warm worker slot and uses standard proxy environment variables when needed.

Multi language selectors use one shared language manifest. Select options may be either primitive values or { value, label, label_i18n_key } objects; label is the stable code/native display string such as yue/中文-粤语, while label_i18n_key points to the localized language name used by the website UI and API documentation.

For image_upscale, submit the unified dimension contract with reference_width, reference_height, and an empty aspect_ratio. reference_width and reference_height must match and can be 2048 or 3072. The legacy target_max_side field is rejected. The worker preserves the input aspect ratio and reports the final width and height in the runtime summary.

MM Audio AIO

MM Audio AIO is available through API Models, Playground, and the Visual site group. It has three task types: text_to_audio, image_to_audio, and video_to_audio. Every task requires a prompt. Text tasks reject media, image tasks require exactly one source image, and video tasks require exactly one source video. Each media input is limited to 100 MB, decoded images are limited to 64 megapixels, and source video is limited to 20 seconds. Original media may exceed 1024 pixels on its longest edge; the Worker decodes it and proportionally normalizes image and video inputs to a 1024-pixel longest edge before inference.

MM Audio AIO uses Fast credits only. Text tasks return one 10-second audio/wav file. Image tasks return one 10-second, 25 FPS H.264/AAC video/mp4 file that combines the source image with generated sound. Video tasks return one H.264/AAC video/mp4 file that preserves the source visuals and follows the decoded source duration, up to 20 seconds. The live catalog remains authoritative for task inputs and outputs.

Readiness requirements

A model group should not be public until these are true:

  • Worker warmup completes before accepting tasks.
  • Segmented timing logs are available for download, inference, upload, and total runtime.
  • Queue and scaling limits are configured.
  • Pricing and credit deduction rules are defined.
  • At least one smoke task has passed in the current release candidate.