AI-driven Virtual Avatars: The new paradigm of customer service

What began as simple representations in early Internet forums has
evolved into more realistic versions with applications in a variety of
industries.

In fact, according to GrandView Research, the global digital
avatar market, valued at $14.34 billion in 2022, is
expected to reach $270.61 billion by 2033. Sectors such
as healthcare, industrial, or marketing are the ones that can obtain more
potential and results, but their benefits can cover many more, we explain
everything!

Extended reality (XR) has ushered in an exciting era of immersive interaction, allowing users to see, hear, and
interact with virtual content as if it were part of their physical
environment.

This paradigm shift to realistic virtual environments naturally
extends our desire for virtual agents that emulate human communication and
interaction. As such, developing such intelligent virtual agents is paramount
for future XR systems, with potential benefits in applications
such as customer services, professional work environments, and video
games.

Agents represent a human figure created by software, while avatars
usually represent a real person. Despite this small difference, both face
several common challenges, such as appearance and movement, control
mechanisms for interactivity and autonomy, gestures, locomotion, interaction,
cooperation, coordination, etc.

Looking back just three years, there were few tools to create ultra-realistic avatars and motion capture
techniques were not available to everyone. The advent of generative AI has changed
this completely.

Generative AI is making it possible to reproduce
human cognitive functions, so AI-powered avatars can hold a
natural conversation with a human being and attempt to show
emotions.

Some of its most important benefits are:

• Better customer experience: virtual
  avatars have been a small revolution in the world of customer service. Thanks
  to AI, virtual assistants are available 24/7, providing a powerful tool for
  creating a more effective experience at a lower cost.
• Assistance: virtual avatars are
  becoming increasingly popular in the field of training and education. They
  are a support, like a private tutor, who can help to review lessons, but who
  is also able to give personalized classes adapted to each student. They are
  also very useful in the healthcare field, where they can help patients and
  reduce delays in processes.
• Personalization: these avatars can
  collect and analyze a large amount of user data. This process facilitates the
  identification of potential customers’ requirements and expectations,
  allowing companies to adopt a personalized approach when interacting with
  their audiences.
• Improved brand image: companies can
  create customized avatars that represent their brand and use them on online
  platforms to increase brand recognition.
• Better representation of products and
  services: proper product awareness is critical to improve
  sales and overall business progress. Virtual avatars allow you to show these
  products and services in a realistic and detailed way, allowing potential
  consumers to learn about each feature.
• Increased engagement: Dynamic
  animations captivate users by making interactions more visually appealing. As
  a result, users are more likely to stay engaged and attentive when avatars
  display more realistic behaviors and movements.

In the constantly evolving landscape of digital interaction, companies need to move towards
innovative solutions that resonate with their audience and differentiate them
in the competitive marketplace.

Digital avatars are undergoing many changes, and several
characteristics are essential to transforming the way companies connect with
their users:

GPT-driven conversations

The integration of GPT into conversations represents a
breakthrough, as it gives avatars the ability to interact
with users in contextually relevant dialogues.

Avatars with GPT technologies are excellent at understanding the
context of a conversation, as they can capture details, infer meanings, and
maintain continuity in dialogues.

Language generation capabilities enable avatars to produce text
that is not only grammatically correct but also contextually appropriate.
This ensures that avatar-generated responses are accurate and mimic human
communication, as well as continually refining their
understanding of users’ language patterns and preferences
as they interact with them.

Their versatility to engage in multifaceted conversations makes
them well-suited for applications ranging from customer service to
educational and entertainment platforms.

Realistic
animations

Realistic animations are a fundamental feature of avatars,
revolutionizing the way users interact with virtual entities. It may seem
simple, but dynamic animations are what “bring avatars to life,” providing
them with fluid movements and expressive actions that elevate the overall
virtual experience.

Dynamic animations allow avatars to perform realistic gestures,
which adds a layer of realism to interactions, enhancing
interactivity. This allows them to recognize the user’s
presence, react to specific commands, or respond to environmental stimuli,
creating an engaging and responsive virtual environment.

Perfect
integration

Seamless integration promises a harmonious merging of virtual
entities with existing platforms, workflows, and user experience. This
ensures that avatars become an integral part of a user’s digital journey
without causing disruption, delivering consistent and unified
interaction across multiple touchpoints.

A seamlessly integrated AI-powered avatar adapts seamlessly to
different platforms: websites, mobile apps, or virtual environments.
Regardless of the space, the avatar maintains consistent functionality and
appearance, ensuring a cohesive experience.

Integration extends beyond surface-level
compatibility, allowing avatars to align with existing
workflows. This synchronization enables a seamless transition
for users, allowing them to interact with the avatar on different devices,
but without losing context.

Multilingual
capabilities

The ability of an avatar to communicate in different languages
completely revolutionizes the way companies can connect with diverse
audiences around the world. Avatars equipped
with multilingual capabilities break down language
barriers, thus extending the reach of virtual interactions to a global
audience.

This capability goes beyond simple translations but encompasses an
understanding of linguistic details, cultural contexts, and regional
variations. These switch seamlessly between languages in real-time based on
user preferences or contextual cues.

Beyond text-based interactions, they support multimodal
communication, including speech synthesis and speech recognition in multiple
languages.

Privacy and
Security

An essential point in virtual avatars is privacy and security,
which are essential to protect users’ sensitive data and comply with
regulations such as GDPR and CCPA.

Avatars use advanced encryption and data
anonymization to ensure confidentiality and security
during transmission and storage. In addition, they implement
strong authentication and regular audits to identify and mitigate
vulnerabilities.

This generates user confidence, as well as compliance, minimizes
the risk of breaches, enhances organizational reputation, and a competitive
advantage over other similar companies.

Predictive capabilities

Predictive nudges optimize interactions
by anticipating user needs and preferences by analyzing
behavioral patterns. These avatars use Machine Learning
algorithms to provide contextual, personalized, and timely reminders that
enhance the user experience.

This increases engagement by guiding users to relevant actions,
improves satisfaction by offering personalized suggestions, optimizes
conversion rates, or creates personalized marketing opportunities aligned
with user interests.

To create realistic avatars, it is crucial to collect accurate information about human form and movement.
Marker-based motion capture, known as “mocap,” is the most reliable method
for this.

This process involves transforming a sparse, raw 3D point cloud
into usable data. Initially, the data is cleaned and labeled by assigning 3D
points to specific marker locations on the human body.

A major challenge in capturing extensive motion
capture data is the labeling process, which, despite
using the best commercial solutions, often requires manual intervention.
Issues such as markers and noise can complicate matters, especially when new
marker sets are employed or when humans interact with the
objects.

On the other hand, facial capture data is vital
for building realistic human models. However data capture is
only the first step in creating virtual avatars, as modeling involves
transforming the captured data into a parametric model that can be
manipulated, sampled, and animated, with a focus on varying human shape and
movement according to different poses.

The next step would involve texturing and
shading the avatar to achieve a realistic look. This includes
applying textures that simulate materials such as skin, hair, and
clothing.

Finally, the avatar is animated and integrated into digital
applications or robotic systems.

With this demand and aspirations for real-time responsive virtual agents, many professionals are trying to achieve
the level of naturalness and realism required. One of them is Plain
Concepts.

To overcome the current limitations of virtual agents, we have
developed an avatar system based on Machine
Learning, capable of interacting naturally with users
using multimodal signals in real-time.

An example is the avatar we have developed for the IFMIF-DONES
project, the new particle accelerator being built in Granada. This
virtual 3D avatar, which can be shared through Microsoft Teams, and which is
being trained with project-specific information, can be used to interact
naturally with users.

We have used proprietary technology for the creation of
interactive 3D avatars, which allows the user to communicate naturally with
the virtual assistant. The avatar is able to listen and
respond in natural language using an artificial voice. The
understanding and generation of coherent answers to the user’s questions are
processed by the most advanced AI technology on the market at the moment,
provided by OpenAI and Microsoft.

The avatar has a realistic 3D appearance, able to gesticulate and
pronounce the answers imitating the human voice, simulating a natural
conversation with the user, which enhances interaction and
engagement.

The AI has been developed on Microsoft Azure and the pre-training
has been done with texts provided by the client, achieving a specific context
of the project. In addition, the application is compatible with Microsoft
Windows operating systems and can be run on any PC or laptop with a mid-range
or high-end graphics card.

One of the main challenges we faced was to get an avatar with all
kinds of facial expressions, for which we relied on Microsoft’s viseme
collection (different positions of the mouth when pronouncing the most
important phonemes) to be able to animate in real-time the avatar’s mouth.
From here, we needed a realistic avatar, based on a realistic look,
expressing all kinds of emotions and reflecting human attitudes.

The goal was to get a clean
interface, so that most of the UI when you are talking to the
avatar, was the avatar itself. For example, we have given it the expression
of “thinking” so that when the server is processing the answer, a simple
“Loading” does not appear, but we are in front of an avatar with an
expression of thinking.

For this, we have done many motion capture tests, achieving very
realistic and natural animations that simulate human facial expressions. As a
DNA chain, to fill in “the gaps” left by Microsoft’s Cognitive Services
animation services, we worked with the Plain Concepts design team to stage
different situations and scenarios that the avatar could face. We mixed
different types of blending with all kinds of animations to make the avatar
feel more natural.

With all this, we put together a “cocktail” of
technologies, where to create the avatar we used:

• Speech-to-Text AI to convert audio files into text transcripts as
  well as recognize speech.
• Neural Voice to reproduce the human voice.
• Our Evergine graphics engine for the
  3D appearance of the avatar.
• Azure OpenAI for response intelligence and text-to-speech
  conversion.
• .NET MAUI for visual interface and compatibility with iOS and
  Android.

And since a picture is worth a thousand words, here you can see
how “Silvia” acts when you ask her to explain the Granada particle accelerator
project in English in detail:

As a conclusion, advances in AI are offering promising solutions
that can captivate users and at a much lower cost than a few years
ago.

At Plain Concepts we merge the latest GenAI tools with our
proprietary technology to bring your ideas to life. Our experts will help you
explore the full potential of intelligent platforms to create a solution
tailored to your specific needs and give a new meaning to your business.
Contact us and we will show you all that the world of digital avatars can
offer you!