Original title: "Virtual Beings - A New Internet-Native Medium for Shared Storytelling"
Original post by Justin McAfee @1kx
Compilation of the original text: Bi Depan@节动BlockBeats
This is a new shared story for internet-native media, and the increasingly virtual nature of our shared social experiences will increase the number of Virtual beings (virtual beings) for decades to come.
Avatars are entities or characters that exist entirely in the digital realm, leaping into our world through social media interactions, shared media such as games and internet-native spaces, and holographic technologies. They not only have virtual avatars manipulated and controlled by humans using motion capture technology, but also AI that fully virtualizes all perception.
Therefore, virtual human encompasses multiple technological trends such as VR/AR, artificial intelligence, high-fidelity CGI art, gaming, social media, and also human social psychology such as social relationships, desire for companionship, cultural animism, Anthropomorphic design and other characteristics.
And Web3 has the potential to further accelerate the emergence of these digital consciousnesses through new creator revenue models, internet-native organizational structures, and crypto-enabled digital ownership.
TL;DR
1) Define and briefly discuss the history and characteristics of different types of virtual humans;
2) review some of the existing technology stacks used in the creation of these avatars;
3) Identify areas where cryptoeconomics and Web3 technologies can improve the creation, management and distribution of virtual humans;
4) Highlight existing well-known projects in the virtual human field;
5) Speculate on the future of Web3 social experience in the era of AI and synthetic media, such as "Generated media", AI generated media, personalized media;
By reviewing multiple aspects of virtual humans, this article hopes to shed light on this concept as a trend that can significantly benefit from the use of the Web3 technology stack.
Types of avatars
There are many types of avatars, and they vary in complexity.
VTubers
According to Virtualhumans, VTubers are usually virtual characters created by humans using a series of consumer-oriented motion capture technology and character creation software, and broadcast their lives live on popular video platforms, such as YouTube, Twitch, Niconico, and Bilibili. .
Originally, VTubers originated in Japan in the mid-2010s and gradually developed into a global phenomenon by the early 2020s. By manipulating virtual avatars using real-time motion capture technology, VTubers post videos on a range of platforms to reach a fanbase drawn to the characters they create.
Way back in February 2010, when visual novel maker Nitroplus started uploading content to their YouTube channel, using their mascot "Super Sonico" to discuss upcoming releases. However, at the end of 2016 the phenomenon reached a tipping point: "Kizuna AI" made its debut on YouTube.
Kizuna AI created the term "Virtual YouTuber" (VTuber for short) for the first time, and with the affinity of answering fans' questions and comments, the distance between them has been drawn into the distance. Within ten months, Kizuna AI had grown her fan base to 2 million subscribers.
After the success of Kizuna AI, the VTuber trend started in Japan and gradually expanded internationally by appealing to anime and manga fans. In addition, the dramatic increase in Twitch viewership during the COVID-19 pandemic in 2020 has further increased the familiarity of VTuber outside of Japan and led to the creation of a separate channel for it on Twitch.VTuber tags; and YouTube is also2020 Culture and Trends ReportIt was listed as a major trend in , highlighting that the concept had grown to 1.5 billion monthly views in October of that year.
Today, VTubers are using the ability to interact with fans, parasocial relationships, digital avatars, and storytelling to create characters that are more interesting and dynamic than human anchors, and more than 16,000 VTubers have millions of followers on social media platforms . Popular ones includeGawr Gura、wait.、Nyanners、Ironmouse、Kiryu Coco、Code MikoandAi Angelicawait.
Gawr Gura's VTuber image of Japan's hololive production
In Web3, NFT is also known for creating a fanatical fan base, community interaction and sense of belonging, so they have a very high fit with VTuber. At the same time, some VTubers have already started experimenting with this technology, and Kizuna AI announced that it will stop updating from February 2022 and will spend more time to further explore the use cases of NFT.
Additionally, the most obvious use cases for VTuber NFTs are the creation of Tokens to control content and Discords, as well as the creation of digital collectibles. But two other, more prominent possibilities may be: "rights management" and "avatar market."
Because fully controllable and manipulable character models can be created by third parties and distributed in the form of NFT, this is an upgrade to the existing market and delegation system, creators can better track the use of their models, and at the same time earn secondary sales income .AnataOne such experimental project launched 1,000 male and 1,000 female Live2D VTuber avatar NFTs with full functionality in the streaming app.
Virtual Influencers
Virtual Influencers are digital personas created in computer graphics software and given a personality defined by a first-person worldview, then accessed on media platforms to gain influence.
Virtual Influencer Imma
The lifestyle of Virtual Influencers is portrayed on social media, interacts with fans through multiple mediums, and is often used by brands as spokespersons for their products. However, unlike normal celebrities, Virtual Influencers are CGI (Computer-generated imagery)-generated characters, so they can be anyone, do anything, and walk anywhere in the digital world. They are not physically limited, can be in two places at the same time, never tire, never fall out of script.
Among them, one of the most famous Virtual Influencers isLil Miquela, a 19-year-old self-proclaimed android pop star who lives in Los Angeles, has modeled for Prada, appeared in a Calvin Klein ad, was interviewed at Coachella and topped the Spotify charts. In 2020, Lil Miquela earned about $12 million for her parent company, Brud.
Virtual Influencers sponsors like Lil Miquela are almost three times more likely to work with them than their peers because they are in control of their identity. While many celebrity sponsors have paid dearly for the mistakes some brands made, that would never happen to Virtual Influencers whose entire lives are scripted.
In addition to ease of control, Virtual Influencers also provide brands with the ability to tell stories across multiple platforms and formats in a way that their human counterparts cannot yet do. Different access points are provided. To understand this method, you can refer to theTravis Scott Concert。
In order to perform in this game, you have to create an avatar for Travis Scott, and you know it's not actually him, it's just an avatar he's playing in the game. Alternatively, Virtual Influencers look the same on all platforms, creating a narrative spell that wouldn't be possible with a physical-to-digital transcription of a real-life celebrity.
By far the most prominent place for brands to engage with Virtual Influencers is in fashion. Virtual Influencers often wear the latest outfits when posing on social media. Their scripted control allows them to create specific lifestyle elements and personalities for their brand. Like Lil Miquela at Prada's FW18/19 show in Milan, Shudu Gram as the digital face of Balmain models, Noonoouri as part of the "Rouge" campaign originally starring Natalie Portman, and Dior's ad It also adds a lot of color.
Currently, major fashion houses such as Balenciaga, Alexander McQueen, Valentino, H&M, Nike, Salvatore Ferragamo, etc. have accepted Virtual Influencers to showcase their products, and Prada, even considered creating its own in-house Virtual Influencers.
In addition, music is also a widely used field of Virtual Influencers:
- Yameii Online's track "Baby My Phone" inspired multiple Tik-Tok co-production trends and mashups, reaching millions of views across hundreds of thousands of videos;
- Kai, a 16-year-old Roblox Virtual Influencer best known for creating the game Splash (which recently raised over $20 million from investors), performed over 100,000 fans simultaneously at 7 different Roblox venues 48 hours;
- K/DA, the virtual K-Pop girl group created by Riot Games to promote new game characters, has 508 million views on Youtube with its viral music video and song "Pop/Stars";
- FN Meka rapped on social media while showing off ultra-rare merchandise like a Starbucks-branded PS5 and AR sneakers.
andLil Miquela、LV4、Rae、Shudu、FNMekaandYameii, and major Web3 studios such as RTFKT have partnered with Virtual Influencers to promote their NFTs.
In the future, the integration of Web3 and Virtual Influencers will be even more exciting, such as being used in events or advertisements, and Virtual Influencers will curate their stories in a decentralized form through DAO, as well as receive and distribute income that flows to the community treasury .
distributed roles
Distributed Characters are digital characters managed by an open, permissionless community that fans use to create music, animations, videos, lore, and more. Distributed actors may be autonomous in some form, such as AI artists, but are ultimately governed by the community. So it's not like Virtual Influencers, it's not managed by having an authority or a dedicated studio.
Currently, one of the best examples of distributed characters is "Hatsune Miku", a Japanese Vocaloid software voice library released by Crypton Future Media in 2007. Vocaloid allows users to synthesize singing voices by inputting lyrics and melodies. Because each Vocaloid has its own characteristics and is actually an independent "singer", they are usually released as "Moe anthropomorphism" (cute anthropomorphic) characters, and the most successful of these is Hatsune Miku.
Originally, Hatsune Miku was released under the "No Remix" license. Since her debut, she has appeared in hundreds of thousands of songs and music videos made by creators and fans from all over the world. And the freeware program MikuMikuDance spawned a boom in user-generated animation, even in live holographic performances attended by fans around the world, while the distributed nature of Hatsune Miku and her core remix culture prompted Crypton to adopt a non-commercial Creative Commons Attribution License.
For example,
For example,BottoIt is a decentralized autonomous AI artist controlled by native governance Token BOTTO holders. Token holders stake $BOTTO to obtain non-transferable voting rights (VP), which are used to vote on 350 AI-generated artworks every week. At the end of each period, the work with the most votes will be auctioned as an NFT on SuperRare, and the proceeds will be used to buy back and burn BOTTO. To date, Botto is one of the highest-paid artists on the market, with total sales of over $2 million.
And because creative AI has the ability to create infinite new works, human curation may play an important role in the new economy. A DAO that manages the curation, distribution, promotion, and governance of distributed characters while benefiting from value creation could usher in a new era of AI or human-generated digital character crafting.
autonomous entity
Autonomous entities are created by simulating AI in machines that are programmed to think like humans and mimic their behavior. It can also be applied to any machine that exhibits characteristics associated with human thinking, such as learning and problem solving.
While the aforementioned three types of avatars are widely controlled by humans in one way or another, however, this is not the final type of avatars. Autonomous entities will use artificial intelligence and other tools to simulate life, including chatbots, NPCs in virtual worlds, and types such as artificial intelligence companions and digital reincarnation.
AI companion
One of the simplest forms of self-governing entities is the AI companion, and the growing trust and acceptance of AI among younger generations seems to herald a future of intimate human-AI entities. An entire business model formed around this idea, likeReplikaSuch companies offer friendly chatbots that interact with their humans throughout the day;Hybriprovides custom 3D model robots to simulate romantic companionship.
Image source:
Image source:KPMG
And a project in Web3 that focuses on AI companions isPetaverse Network, which draws on a long tradition of virtual pets from Tamagotchi to Nintendogs. Petaverse Network seeks to create an interoperable digital standard for virtual pets, publishing their personalities on-chain so they can live forever. This personality represents the pet's data layer, which is updated over time as users interact with it in games, AR, and VR. And other projects will also be able to build on top of this data layer, reinterpreting pets for their own virtual worlds and games.
digital reincarnation
Extending the AI companionship even further, AI could be trained on the digital footprints of individuals to replicate their behavior after death, a so-calledDigital reincarnation. With enough text messages, social media posts and videos as training data, a neural network can learn to mimic the speech and mannerisms of the dead, while presenting itself as a realistic CGI model or chatbot.
Recently, it was reported that Joshua Barbeau used software to create a surreal chatbot powered by GPT-3, recreating the experience of talking to her late fiancé by training it on text messages from her late fiancé. But due to the sensitive nature of digital reincarnation, rights management is critical, and encrypted NFTs that require pledges to access may be the best solution to ensure that these digital copies are not used elsewhere on the Internet.
Source: San Francisco Chronicle
However, autonomous entities are not limited to AI companions or digital reincarnations, but can also be agents in virtual worlds. An obvious problem with many of the virtual worlds we explore today is that they still feel empty, and populating them with highly intelligent NPCs would be one way to bring these environments to life. One can imagine the following scenarios:
- Decentral Games' virtual casino is equipped with a highly intelligent chatbot;
- Browse the Cryptovoxels store that sells digital goods 24/7 and has AI NPCs living there;
- Use an anthropomorphic trading robot with a distinctive personality;
- On-chain creatures that use oracle data feedback and adapt to external variables; based on user behavior across multiple games,
- Use game characters that evolve morally over time.
So the possibilities of a virtual human with artificial intelligence in the digital realm will be endless.
Virtual Human Technology Stack
Virtual Humans generally run and operate on the following various technical and creative infrastructures.
AI
The degree of autonomy of virtual humans varies, from fully controlled by humans (such as Vtuber, most Virtual Influencers), to natural language processing (NLP), computer vision and used for memory storage, emotion, motivation, etc. Some of the most common tools in this field are OpenAI's GPT-3, a large language model that is highly accurate at describing conversations with actual entities, or machine learning algorithms that learn how to to update the underlying dataset.
While many of today's autonomous virtual humans run on centralized AI infrastructure, two prominent projects in Web3 have taken different approaches to rebuilding AI on-chain.
Alethea AI, the first project based on the "iNFT standard", was listed at Sotheby's last year.478,Alice is sold for $800.The iNFT hashes the GPT-3 personality hint into the NFT's metadata, which is subsequently decrypted on the Noah's Ark platform and used to generate a GPT-3 response to user input. These iNFTs can be trained through multiple interactions with the user, progressing in different levels of intelligence over time. So users can create iNFT versions of their favorite digital assets by pairing any NFT with a personality pod and training it with a combination of user interaction and $ALI token staking.
Altered State MachineA different approach is used to achieve a similar goal. An agent in ASM starts with a table (an image or model of the agent) and a brain (a base NFT with a "genome" matrix, a random set of base values, and a memory tree that records state changes). GPU cloud computing providers use their hardware to run training algorithms for specific ASM applications such as popular games or DeFi trading bots, similar to how mining works with PoW blockchains. The output of these training sessions is called a memory and is recorded in a memory tree in ONNX format, modifying the value matrix to improve the skill set for that particular application.
The first proof of concept for ASM isAIFA, an AI soccer game that uses a combination of Unity, IPFS, and the example Training Gym. However, ASM hopes to expand the application of its agents beyond, and it has proposed some conceptual ideas including DeFi transaction agents, game role agents, chatbots and digital assistants.
So the design space created by on-chain AI agents remains largely unexplored, and it will be exciting to see how this area develops. Importantly, the shift from proof-of-work to proof-of-stake means that large numbers of GPUs will find alternative uses, so decentralized AI models using GPUs for mining may also be the next frontier.
3D Modeling and Animation
At the heart of all iterations of virtual humans is the use of 3D modeling, animation or motion capture techniques. While virtual humans come in a variety of graphic aesthetics and forms, all require basic models to be assembled for animation or movement. Markets and repositories for these models already exist, and NFTs can provide additional avenues for their financialization.
wait.Unity、Unreal Engine、Blender、iCloneandFaceRigwait.
While some avatars don't try to look photo-realistic, those that do require a lot of work, designers often struggle to overcomeUncanny valley(uncanny valley), which is the innate sense of unease and disgust humans feel when viewing an imperfectly rendered human face. But the popularity of the uncanny valley effect may diminish over time as a new generation becomes more accustomed to interacting with CGI and humanoid robots, though for now, rendering of humanoid characters must account for this physical aversion.
The key constraints of realistic rendering are determined by computer processing power, such as the rendering of high-fidelity faces with sufficient detail and deformation control, as well as mesh 3D models with texture images and specular maps, all of which are intensive.
andArtisant,as well asThe Fabricant、The Dematerialized、DIGITALAX、Tribute BrandandRTFKTIf such digital fashion companies continue to adopt Web3 technology, they will further occupy the market share in the virtual space. And with the increasing popularity of virtual characters, digital fashion may also grow in tandem.
Additionally, the difficulty of creating avatars will help create a vibrant market for 3D models, with creators selling the rights to their models to those who wish to use them in digital art, games, movies, and more. And Web3 unlocks additional opportunities here: NFT 3D models that can be resold, secondary sales fees go back to the original creator, ultra-smooth payment flow can control the model while using it in the virtual world, permissions through encrypted files Control, must be pledged to unlock, etc.
For animation, the 3D mesh will be bound to the skeleton, which can then be animated through code or user action. At present, experiments combining animation and gestures with NFT have begun,HEATrepresents the forefront of this movement, hoping to create a market for animations that can be applied to 3D models. By retargeting .glb and .gltf files to the skeletal structure of the 3D model, HEAT will be able to create interoperable motion files for any standardized avatar file in the virtual world.
virtual human studio
The size of the team needed to build a virtual human depends largely on its complexity. anyone can useVRoid Studio、FaceRigand streaming setup to create a simple VTuber character, but Virtual Influencers or Autonomous Entities may require expertise in AI, animation, speech synthesis, modeling, content creation, and more.
Avatar studios typically have teams dedicated to CGI art, photography, animators, storytellers, social media planners, fashion strategy, business development, PR, financial management, merchandising, and more. These studios can focus on the management of a single character, or they can focus on creating a large number of characters. For example, The Diigitals Agency behind Shudu Gram manages a total of 7 Virtual Influencers; Lu of Magalu is supported by one of the largest e-commerce enterprises in Brazil; SuperPlastic manages Janky & Guggimon, Dayzee & Staxx, Kranky, ShuDog, etc.
It is worth mentioning that,
It is worth mentioning that,CULTUR DAOIt is the most famous decentralized virtual human studio, calling itself Web3 Pixar. By controlling Discord and content with tokens, a group of artists, technologists, and evangelists use shared economic incentives to drive innovation in the field. Meanwhile, committees were formed around avatars, AI, speech synthesis, story creation, and experimentation with existing AI NFT infrastructure.
Recently, their hackathon has also produced many interesting projects, including Decentraland AI Chatbots, virtual human ballet, talking Cryptopunks, AI character memory test, and a virtual pet like Kittygotchi.
synthetic social media
synthetic mediaAI-generated personalized media content is proliferating, and avatars are a further extension of this social media layer. While virtual humans may still feel alien right now, it's not hard to imagine that in a generation or two they will become commonplace and widely accepted by society.
Deepfakes, social media bots, and anthropomorphic smart devices are already blurring the lines between humans and virtual beings, and this will continue as younger generations interact with virtual humans more frequently. Statistics show that almost 15% of the audience of Virtual Influencers are young people between the ages of 13 and 17, which is twice the average for ordinary influencers.
So the initial stage of this kind of virtual human synthetic social media has arrived, such asnews of marriage between a human and a virtual person、Hybriand other dating apps, dedicatedvirtual companion platform, and Virtual Influencers who have more influence than humans, our growing trust in artificial intelligence, etc.
Original link
IOS
Android