This Image-to-Video workflow converts a single reference image into a coherent animated video sequence using LTX-2 and its unified audio-video latent architecture. The input image acts as the first structural frame, anchoring identity, composition, and visual continuity while motion, timing, and audio are generated over time.
The pipeline uses an automated prompt enhancement stage (Gemma-based) that analyzes both the reference image and the user’s raw prompt to produce a concise, action-focused video description. The enhanced prompt emphasizes temporal actions, camera behavior (when explicitly requested), and audio continuity, while intentionally avoiding repetition of visual details already present in the input image to prevent scene breaks.
Video and audio are generated jointly in AV latent space through a two-stage sampling process, allowing finer control over temporal stability and visual fidelity. The workflow exposes explicit controls for:
- Frame count and frame rate
- Image strength, defining how strongly the animation adheres to the reference image
- Optional camera LoRAs, enabling guided camera motion (push, pull, pan, etc.) without hard-coded trajectories
- Optional distilled or detail LoRAs for stylistic or structural refinement
Once sampling is complete, the combined AV latent is:
- Separated into video and audio latents
- Decoded using a spatio-temporal tiled VAE to preserve detail and temporal consistency
- Recombined and exported as a final H.264 MP4 video with synchronized audio
This workflow is designed for:
- Controlled, repeatable image-to-video generation
- Strong temporal coherence with minimal flicker or scene drift
- Native audio generation aligned with visual action
- Advanced cinematic or narrative pipelines built entirely within ComfyUI