Read this article in Estonian and Russian
A young man sits on a cloud and reads a book, a cute Dalmatian jumps along the sunny house window sills, the Chinese New Year parade engulfs the city street, and an archaeologist digs a chair out of desert sand. These videos posted on the internet depict events that never actually happened. They are the first public videos created by OpenAI’s recently unveiled video generation tool, Sora.
Sora creates videos up to one minute based on user descriptions, much like how ChatGPT responds to input with written answers and DALL-E generates images.
The new technology allows for the creation of more elaborate videos with significantly fewer resources, but it may also increase the spread of misinformation on the internet and enhance government propaganda, further blurring the already weak boundary between real and fabricated content.
Kristian Hammond, a computer science professor at Northwestern University, remarked, “The clarity of truth that we thought was preserved with photography and video is gone. We have inadvertently built a world of propaganda engines.”
In response to ABC News’ question about what measures are being taken to prevent Sora’s misuse, OpenAI referred to its website.
“Before making Sora available in OpenAI products, we are implementing several important safety measures,” the company states on its website. “We collaborate with red team members – experts in areas such as disinformation, content that incites hatred, and bias—who test the model as a countermeasure.”
The company plans to implement some of the safety features used in the DALL-E image generator, including a tool that checks text excerpts to ensure they do not violate rules regarding “extreme violence, sexual content, incitement to hatred imagery, and celebrity likenesses or the use of others’ intellectual property.”
It is challenging to evaluate a demo product that has not yet been made publicly available, but despite this, caution is advised regarding the potential misuse of the video generator and the challenges associated with implementing fully effective protective measures.
Sam Gregory, Executive Director of Witness, highlighted that “realistic images of events affect people’s assumptions about what is happening in the real world and can be used to deceive people.” Witness aims to use video and technology for people’s protection and human rights.
The risks associated with AI-generated content have been particularly prominent in recent weeks. Falsely generated, sexually explicit images of pop star Taylor Swift circulated on social media at the end of January, garnering millions of views. A fake robocall impersonating President Joe Biden unsettled people during last month’s New Hampshire primaries.
Sora, like other generative AI products, is trained on web data, making it susceptible to widely reproduced biases, such as racial and gender stereotypes.
In addition to moderating video clips and their resulting content, OpenAI plans to implement a “detection classifier” capable of identifying when Sora has produced a video. The website notes that it also includes a popularly recognized digital watermark, essentially meaning a digital watermark.
While such precautions are commendable, videos may still be replicated or altered to remove these labels.
“People try to bypass installed safeguards,” added Hammond. “It’s an arms race.”
Screenshots are from sources used in the story.
