It was a peaceful Sunday morning in December 2021.
Moon Jae-yoon, a 21-year-old who loves to read and draw cartoons, got up from her bed and scrolled down news articles to see what she has missed overnight.
Her fingers tapped an article about a popular web novel’s characters being recreated into virtual idols. The article said that these idols will debut in metaverse platforms in South Korea. Moon still remembers how shocked she felt to read the news.
“I felt a bit confused by the news at first,” Moon said. “Who would have imagined those fictional characters to be realised in a virtual space thanks to the metaverse trend?”
The article that Moon read turned out to be a mere rumour spread by unreliable sources later. However, the development of metaverse idols is really under progress in South Korea.
With the soaring interest in metaverse technology nationwide, the K-Pop industry is seeking a chance to jump on the virtual reality craze. Some artificial intelligence developers are “building” and “training” their virtual K-Pop artists to steal people’s hearts.
Metaverse Idols
Metaverse, which has now become a “buzzword” in the tech industry, is often understood as a digital universe where users can interact with each other as they would in the real world.
Some metaverse platforms allow their users to grab a coffee at a virtual cafe or visit art galleries. Virtual spaces like ZEPETO developed by NAVER Z and Fortnite are some of the representative metaverse platforms.
The concept of this virtual space became more popular as people’s outdoor activities were restricted due to the pandemic, which also impacted the K-Pop industry. As people cannot attend any offline meet-and-greet events or concerts, artists started to make their events “online” and “virtual”.
In September 2020, BTS, a popular Korean boy band, for example, premiered the choreography of their song “Dynamite” on Fortnite’s Party Royale, where players can get together to enjoy cultural activities.
Some developers try to go “beyond” that, by creating AI-generated virtual K-Pop idols to perform in metaverse platforms. These virtual idols will sing and dance to their songs as human performers do in the digital space.
Korean game developer Netmarble recently founded an entertainment company “Metaverse Entertainment” to build virtual reality platforms and in-game content, such as virtual idols.
The company will also partner with the internet giant Kakao Corp. to create a four-people virtual idol group to perform in metaverse soon, according to the reports. The group, known as MAVE, will consist of AI characters and make its official debut this year.
Alongside the tech giants, startups are also preparing to launch or have launched AI idol groups on metaverse platforms.
Deep Studio, an AI software developer based in Seoul, is planning to debut a four-member band named YOURS this year. Their official launch date is not out yet, but their social media accounts are already followed by more than 200,000 people.
Last March, Pulse 9, also an AI developer, took the internet by storm with their three-minute-long music video featuring virtual idol girl band titled “ETERN!TY”. The video, which now has gained almost 900,000 views, shows 11 girls singing to their original song “I’m Real”.
One of their members DAIN released another song “No Filter” in August 2021, and gained over two million views as well.
How Metaverse Idols Are Created
Creating a metaverse idol can utilise many different techniques, but one of them is synthesising computer-generated faces and real human bodies. To make their faces look “realistic” from any angle, the developers apply graphics onto actors to superimpose their digital idols.
The technique itself is not quite different from the synthesis process of deep fakes. If different, it is that the developers make the computer graphics from scratch based on a collection of people’s faces.
For example, the footage of Pulse 9’s ETERN!TY is enacted by actual human actors whose faces are replaced by computer-generated graphics. The developers of Pulse 9 created hypothetical faces using the database of “good-looking” people.
Their automatic system makes various looks from different perspectives and angles with those faces. After that, the company’s deep-learning model learns these graphics and apply on the actors.
“We previously opened a poll to choose the best-looking individuals from a pool of computer-generated people,” Heo In-kyung, a public relations manager of Pulse 9, told 4i Magazine. “From there, 11 girls were shortlisted to form a virtual girl band ETERN!TY.”
Kim Hyeong-seop, an AI developer at Pulse 9, says that the members of ETERN!TY have more human-like, realistic forms compared to other AI-produced characters out in the market.
“Some of the other AI-built characters take on cartoon-like graphics,” Kim told 4i Magazine during the interview. “But the base of our development is to make more realistic, human-formed idols with the assistance of deep-learning technology.”
Pulse 9 plans to create voices and bodies for the band’s members later this year, transitioning their synthesised graphics to fully computer-generated humans.
“The system will soon learn data sets of voice as well this year,” Kim said. “We are looking forward to creating body movements through the AI learning process soon.”
Becoming a Fan of Metaverse Idols
With their human-like faces and forms, these metaverse idols are growing popular as of late.
Pulse 9 says ETERN!TY is gaining momentum in and outside of South Korea. The band even has a fan club called “Eternal”, which became a community where people can share the band’s photos and clips with each other.
“When ETERNT!Y’s first music video was out, we saw many comments criticising the looks of our band,” Heo said. “But as our graphics and interactions with the fans improved over time, people started to show more positive responses than before.”
Explaining why these idols are becoming more popular than before, Ryu Ki-hyun, CEO of Deep Studio, says being a fan of someone can work as motivation in life.
“We wanted to make people feel less lonely with our metaverse idols,” Ryu was quoted as saying by The JoongAng Daily. “Many people from Generation Z don’t like idols simply because they are good-looking. Those idols are the source of life energy and motivation for them.”
Future of Metaverse Idols
Some say the success of these metaverse idols may depend on whether they look realistic “enough” without creating the Uncanny Valley effect. When graphics or automated mechanics resemble human appearances but not exactly like “humans”, people may feel uncomfortable in response to their looks or movements.
Moon says it is important to make metaverse idols evoke a sense of familiarity and empathy if they want to come across as “likeable”.
When she heard the rumour about the web novel’s characters becoming virtual idols, she first worried whether the characters will look and act the same as the novel described them.
“Fans will be disappointed if the idols look different from what they have imagined by reading the novel, or when the graphics fail to mimic humans,” Moon said. “Graphics should be meet the readers’ level of expectation.”
When asked about the future of AI idol bands, Kim from Pulse 9 said the bands will continue to evolve with the development of technologies.
“For our band, we have been changing the definition of our graphics, advancing and adjusting the algorithms (since the debut),” Kim said. “AI idols may evolve into even better forms in the future with the help of technologies.”
Ryu from Deep Studio says we’ll get to see more metaverse idols shortly.
“I believe the age of metaverse, where we can do our social and economic activities in a virtual setting, will arrive soon,” he said.
“The core value of metaverse is creating ‘another me’ who is better than my (real-world) self. In metaverse, there will be human-formed avatars who can do so much more than real-world humans can.”
