By Sebastien Hinse and Jonathan Dankoff
As seen in the first two articles in the series (Article #1, Article #2), Ubisoft gathers data about player behaviour through telemetric systems to analyse and ultimately improve the overall experience of our customers.
This article will reveal more details on the functional challenges of doing analysis on game telemetry data and offer insights on the technical solutions available to game developers and analysts at Ubisoft. Also, by popular demand we have added a little extra surprise at the end for you.
In most industries, the goal of data analysis is to highlight useful information in order to support decision making. Whether trying to test existing hypotheses, for example in the analysis of the constraint system in Assassin’s Creed, or trying to discover new patterns in the data using exploratory analysis, data analysts use a specific workflow to reach their objectives. The software solutions we offer here at Ubisoft are tailored to that workflow.
Data exploration
The goal of data exploration is to allow analysts and designers to sift through large quantities of data in an attempt to reveal meaningful information, such as patterns, correlations, distributions, or even to identify outliers and explain them.
In order to do this the data must provide accurate contextual information, for example: the player’s current state, mission, and equipment.
By taking this data and visualizing it in an easily comprehensible way, it allows us to compare it to initial design intentions, or to simply examine trends and provide valuable analysis to the development teams.
To achieve this, game analysts need a responsive and interactive analysis experience that helps to find meaning in the data and allows them to iterate quickly with successive “what if” questions, in full autonomy.
The following example is from Assassin’s Creed: Revelations multiplayer, and shows the flexibility and versatility of our analysis tools when it comes to plotting data and exploring it in real-time:
This graph plots the average final score earned in a match against the number of kills the players make. The size of the bubble represents the player’s experience level, and the color shows the multiplayer game mode.
While this may seem like an eyeful at first glance, there are several relatively easy observations to be made:
- The modes are generally well balanced for scoring despite the different gameplay styles and the varying focus between achieving team objectives and getting kills.
- Chest Capture and Steal the Artifact stand out as the modes that offer the highest possible scores for players with a lower number of kills, while the Assassinate mode makes it a little more difficult to get a high score, requiring players to get a lot of kills.
- Corruption, Wanted and Manhunt all have relatively tight grouping, which tells us those modes produce tight games independent of the players levels.
- The bubbles increase in size as they move towards the top, showing that players become better at getting higher scores as they gain experience.
Those are just examples of the type of analyses that help developers adjust the game using title updates, or plan changes for the next installment of the game. This type of ad-hoc data exploration is an essential tool to discover new interesting facts about the game that were not necessarily obvious avenues of investigation when we set out.
Geographic representation
Whereas a typical business would look for sales data divided by country and zip code and represent those on a world or regional map, video game analysts need a way to represent their data in a representation of their virtual worlds, i.e. the maps of the game.
Our in-house tool, DNA Viewer, was inspired by best-of-breed solutions like Microsoft Pivot Viewer, Google Maps and Tableau Software and adapted to the reality of game telemetry. This tool allows us to easily create filtered data sets which are then plotted on the world map of the game, with visualizations and filters specifically tailored to the needs of our game analysts and designers.
This is particularly useful when using mixed-method research. In the following example, user research team members noted that some players were not taking the appropriate path which led to a descent via parachute, but instead descending by an unintended alternate path which bypassed the tutorial and was less fun. It is critical that players do not get lost this early in the game to keep them in Flow, and ensure they get the best possible experience.
The staff had difficulty pinpointing exactly where the players were able to go, and so turned to the viewer tool, which clearly illustrated their paths.
This image clearly shows six of the eight players parachuting as intended, but two other players making their way down via an unintended path.
Thankfully the tool is able to display both 2D overview maps, as well as 3D representations of the whole space. The 3D maps are taken directly from the developer’s production tools such as 3DS Max and integrated into our viewer, offering a perfect reproduction of the game world, with our data laid over it.
The level designer was able to easily fix the area to ensure that the players took the proper path and used the parachute as intended. This resulted in a more streamlined and enjoyable experience for the player.
Final words
As you saw in this series of three articles on game telemetry, Ubisoft has invested heavily in the use of telemetry as a tool to improve our games. As such, we are continuously improving and developing new methods and technologies to better serve the development teams who use the information gather to create better games. Some upcoming projects include a new version of the viewer with better filtering and data exploration capabilities, a central database to collect all sources of data, and other exciting new projects. These new technologies will be used for the upcoming Assassin’s Creed 3, as well as other future projects.
These tools also allow us to provide you with these three wallpaper images from AC Brotherhood and AC Revelations. We hope you enjoy them. Thanks for reading. As always, please don’t hesitate to send me any questions or comments.
