Project Paidia: People and Artificial Intelligence playing together

Up until now, in the world of video games, Artificial Intelligence (AI) has been used to create non-playing characters (NPCs or agents) that are capable of winning over their human opponents. But what if we could teach NPCs to collaborate with us?

This is the starting point of the Paidia project, a research work between Microsoft Research Cambridge and Ninja Theory in which we participated using Evergine to create a web videogame that shows this technology.

AI playing as a team

The NPCs are the soul of many videogames, and new AI techniques make it possible to create characters that behave organically and respond naturally to the players.

By using learning by reinforcement, Artificial Intelligence opens the possibility to create agents that can collaborate as a team with human players, taking different roles and adapting their way of playing as the situation in the game changes.

This learning technique also allows one to train an agent in a video game (environment) in an autonomous way. The agent makes decisions in the game and is rewarded according to the result of their actions. As it happens with human learning, the agent learns from its own experience as the game progresses, improving little by little.

By training several agents at once, they can learn through experience how to work as a team to achieve the goal (win the game) most effectively, making coordinated strategic and tactical decisions.

An interesting advantage of applying this technique to video games is that trained agents, based on their own experience, can adapt to our game and act very creatively. That means that perhaps from our point of view their decisions are quite unconventional, but can be very effective nonetheless. This in turn will allow human players to learn new tactics or improve others.

As a proof of concept, the Paidia project team trained an agent in a simple environment where two players must collaborate. There is a square space with three circular areas, and the goal is to activate two of these circular areas by positioning ourselves on top of them. The catch is that in order to activate them, another player must be positioned on top of the third one at the same time.

Project Paidia

The video game

At Plain Concepts, we have developed a video game that uses the results of the Paidia project to show the first advances in collaborative AI in video games.

This game is designed to quickly test the collaboration between two players – you (white), and the trained agent (blue). It takes place in a closed room inside a spaceship and you must collaborate in order to escape. While one player presses the switch (red button), the other must activate the yellow and blue platforms to open the door and escape.

To be able to play from a web browser, we first had to optimize the agent by reducing information from the neural network that is only used during training. That way we get the model to run in real-time on any browser.

In addition, we had to create the logic of the game for it to adapt to the environment where the agent had been trained so that the agents could play in it. To do this, we used Evergine, which allows us to create optimized executables for both PC and Web using WebAssembly and WebGL. Although at Plain Concepts we prefer to work with WebGL 2.0, in this game we adapted the graphics library to WebGL 1.0 to support Apple devices.

Finally, the communication between the video game and the NPC is done through a Javascript API for which we have used onnxjs. This API receives as input the “state” of the game, executes the AI, and responds with the action that the agent wants to take next.