Less than a decade ago it took a huge VFX team months of work to put Peter Cushing’s face on to Huy Henry’s in Rogue One: A Star Wars Story. Now AI seems to be able to do something comparable in a matter of minutes.
While the results still lack some polish, people are raving about the abilities of the Chinese tech giant Tencent’s new open-source AI video model, HunyuanVideo. One user has dropped a demo showing attempts to put Keanu Reeves into a scene from Apple TV’s hit series Severance, and the effort has quickly taken off on TikTok and other platforms (also see our roundup of the best deepfakes).
The video above was shared by the TikTok user @allhailthealgo. Unlike previous deepfake technology, it was made using video to video, mapping a new actor’s face directly over existing footage.
Hunyuan allows video synthesis with full-body simulation and has the benefit of rapid training on minimal data using low-rank adaptation (LoRA) models. It also appears to have no censorship to prevent the use of celebrity likenesses.
Tencent claims that HunyuanVideo is the most parameter-rich and high-performce text-to-video model available in the open-source domain. With 13 billion parameters, it is capable of generating videos that exhibit high physical accuracy and scene consistency. It boasts rich semantic expression, completing sequential actions in one go, strong complies with physical laws – an issue for much AI video – and the ability to breaking single-camera movements for integration of director-level camera work.
Keanu looks a bit wonky, but many observers have been blown away by the results, which suggest the model has a lot of potential for VFX work. However, the lack of censorship is certain to increase the fears about malicious use of deepfakes to spread fake news.
AI video also remains controversial in terms of quality and ethics Coca-Cola’s AI Christmas ad is one of the latest example of a brand receiving a public backlash as a result.
HuanyuanVideo can be found on Hugging Face and GitHub
