There is Microsoft Research Asia unveiling A new experimental AI tool called VASA-1 can take a still image of a person – or a drawing of a person – and an existing audio file to create a lifelike talking face out of them in real time. It has the ability to create facial expressions and head movements for an existing still image and appropriate lip movements to match a speech or a song. The researchers uploaded a ton of examples to the project's page, and the results look good enough to fool people into thinking they're real.
While the lip and head movements in the examples can still be seen as somewhat robotic and out of sync upon closer inspection, it's still clear that the technology can easily and quickly be abused to create deepfake videos of real people. The researchers themselves are aware of that possibility and have decided not to release “an online demo, API, product, additional implementation details, or any related offering” until they are confident that their technology will be used “responsibly and in accordance with the law.” .” However, they did not say whether they plan to implement some safeguards to prevent bad actors from using it for nefarious purposes, such as creating deepfake porn or disinformation campaigns.
The researchers believe that their technology has many benefits, despite the potential for abuse. They say it could be used to increase educational equity, as well as improve accessibility for those with communication challenges, perhaps by providing access to an avatar that can communicate for them. It could also provide companionship and therapeutic support for those in need, they said, suggesting VASA-1 could be used in programs that offer access to AI characters that people can speak to.
According to paper Released with the announcement, VASA-1 was trained on the VoxCeleb2 dataset, which contains “over 1 million utterances for 6,112 celebrities” taken from YouTube videos. Although the tool was trained on real faces, it also works on artistic photos like the Mona Lisa, which the researchers amusingly combined with an audio file of Anne Hathaway's viral rendition of Lil Wayne. paparazzi. It's so enjoyable, it's worth a watch, even if you're skeptical of what good this kind of technology can do.
This article contains affiliate links; If you click on such a link and make a purchase, we may earn a commission.