HunyuanCustom Launches Single-Image Video Deepfakes with Audio and Lip Sync Capabilities

<div id="mvp-content-main">
    <h2>Introducing HunyuanCustom: A Breakthrough in Multimodal Video Generation</h2>
    <p><em><i>This article explores the latest release of the multimodal Hunyuan Video model—HunyuanCustom. Due to the extensive scope of the new paper and certain limitations in the sample videos found on the <a target="_blank" href="https://hunyuancustom.github.io/">project page</a>, our coverage here will remain more general than usual, highlighting key innovations without delving deeply into the extensive video library provided.</i></em></p>
    <p><em><i>Note: The paper's reference to the API-based generative system as ‘Keling’ will be referred to as ‘Kling’ for consistency and clarity.</i></em></p>

    <h3>A New Era of Video Customization with HunyuanCustom</h3>
    <p>Tencent is launching an impressive new version of its <a target="_blank" href="https://www.unite.ai/the-rise-of-hunyuan-video-deepfakes/">Hunyuan Video Model</a>, aptly named <em><i>HunyuanCustom</i></em>. This groundbreaking model has the potential to render Hunyuan LoRA models obsolete by enabling users to generate 'deepfake'-style video customizations from a <em>single</em> image:</p>
    <p><span style="font-size: 10pt"><strong><em><b><i>Click to play.</i></b></em></strong><em><i> Prompt: ‘A man listens to music while cooking snail noodles in the kitchen.’ This innovative method sets itself apart from both proprietary and open-source systems, including Kling, which poses significant competition.</i></em>Source: https://hunyuancustom.github.io/ (Caution: resource-intensive site!)</span></p>

    <h3>An Overview of HunyuanCustom’s Features</h3>
    <p>In the video displayed above, the left-most column showcases the single source image provided to HunyuanCustom, followed by the system's interpretation of the prompt. Adjacent columns illustrate outputs from several proprietary and open-source systems: <a target="_blank" href="https://www.klingai.com/global/">Kling</a>; <a target="_blank" href="https://www.vidu.cn/">Vidu</a>; <a target="_blank" href="https://pika.art/login">Pika</a>; <a target="_blank" href="https://hailuoai.video/">Hailuo</a>; and the <a target="_blank" href="https://github.com/Wan-Video/Wan2.1">Wan</a>-based <a target="_blank" href="https://arxiv.org/pdf/2504.02436">SkyReels-A2</a>.</p>

    <h3>Sample Scenarios and Limitations</h3>
    <p>The following video illustrates three key scenarios essential to this release: <em>person + object</em>; <em>single-character emulation</em>; and <em>virtual try-on</em> (person + clothing):</p>
    <p><span style="font-size: 10pt"><strong><em><b><i>Click to play</i></b></em></strong></span><em><i><span style="font-size: 10pt">. Three examples edited from supporting materials on the Hunyuan Video site.</span></i></em></p>

    <p>These examples highlight a few challenges, predominantly stemming from the reliance on a <em>single source image</em> instead of multiple angles of the same subject. In the first clip, the man keeps a frontal position, limiting the system's ability to render more dynamic angles accurately.</p>

    <h3>Audio Capabilities with LatentSync</h3>
    <p>HunyuanCustom utilizes the <a target="_blank" href="https://arxiv.org/abs/2412.09262">LatentSync</a> system for synchronizing lip movements with desired audio and text inputs:</p>
    <p><span style="font-size: 10pt"><strong><em><i>Features audio. Click to play.</i></em></strong><em><i> Edited examples of lip-sync from HunyuanCustom's supplementary site.</i></em></span></p>

    <h3>Advanced Video Editing Features</h3>
    <p>HunyuanCustom offers impressive video-to-video (V2V) editing capabilities, enabling a segment from an existing video to be masked and intelligently replaced with a subject specified in a single reference image:</p>
    <p><span style="font-size: 10pt"><strong><em><i>Click to play.</i></em></strong></span><em><i><span style="font-size: 10pt"> Only the central object is targeted, while the surrounding area adapts accordingly in a HunyuanCustom vid2vid transformation.</span></i></em></p>

    <h3>Key Innovations and Data Pipelines</h3>
    <p>HunyuanCustom is not a complete overhaul of the existing Hunyuan Video project but rather a significant enhancement designed to maintain identity fidelity across frames without relying on <em><i>subject-specific</i></em> fine-tuning techniques.</p>
    <p>The model is based on the existing HunyuanVideo foundation and supports various datasets compliant with <a target="_blank" href="https://www.unite.ai/the-new-rules-of-data-privacy-what-every-business-must-know-in-2025/">GDPR</a>, including <a target="_blank" href="https://arxiv.org/pdf/2412.00115">OpenHumanVid</a>.</p>

    <h3>Performance Metrics and Comparisons</h3>
    <p>In rigorous testing, HunyuanCustom has demonstrated superior ID consistency and subject accuracy, as evidenced in a performance evaluation comparative to competitors, indicating a strong positioning in the video customization landscape:</p>
    <div id="attachment_217329" style="width: 951px" class="wp-caption alignnone">
        <img loading="lazy" decoding="async" aria-describedby="caption-attachment-217329" class="wp-image-217329" src="https://www.unite.ai/wp-content/uploads/2025/05/table1.jpg" alt="Model performance evaluation comparing HunyuanCustom with leading video customization methods across various metrics." width="941" height="268" />
        <p id="caption-attachment-217329" class="wp-caption-text"><em>Model performance evaluation comparing HunyuanCustom with leading video customization methods.</em></p>
    </div>

    <h2>Conclusion: HunyuanCustom's Impact on Video Synthesis</h2>
    <p>This innovative release addresses some pressing concerns within the video synthesis community, particularly the need for improved realism and lip-sync capabilities, and establishes Tencent as a formidable competitor against existing frameworks.</p>
    <p>As we explore HunyuanCustom's potential through its diverse features and applications, its impact on the future of video generation and editing will prove invaluable.</p>
</div>

This version has been carefully structured for clarity, SEO optimization, and user engagement while preserving the essential information from your original article.

Here are five FAQs regarding HunyuanCustom’s single-image video deepfake technology that includes audio and lip sync:

FAQs

1. What is HunyuanCustom’s Single-Image Video Deepfake Technology?

   • Answer: HunyuanCustom’s technology allows users to create high-quality deepfake videos from a single image. This means you can generate realistic video content where the subject’s facial expressions and lips sync with audio input, offering a seamless experience for viewers.

2. How does the lip synchronization work in the deepfake videos?

   • Answer: The lip sync feature uses advanced algorithms to analyze the audio input and match it with the phonetic sounds associated with the mouth movements of the subject in the image. This creates an authentic impression, making it seem like the individual is actually speaking the audio.

3. What types of audio can I use with the single-image deepfake videos?

   • Answer: Users can utilize a variety of audio sources, including recordings of speeches, music, or even custom voiceovers. The technology is compatible with different audio formats, allowing for versatility in content creation.

4. Are there any ethical considerations when using deepfake technology?

   • Answer: Yes, ethical usage is crucial. Users should ensure that they have the consent of the person whose image is being used, and the content should not be misleading or harmful. Misuse of deepfake technology can lead to legal implications and damage reputations.
5. Can I customize the deepfake output, such as changing backgrounds or adding effects?
   • Answer: HunyuanCustom allows for some customization of the deepfake videos, including background changes and the addition of special effects. This enables users to create more engaging and unique content tailored to their specific needs.

Source link