Using player analytics for game design

Our new game/research project, Taxi Dash, was quietly released on the Nook and Amazon Appstore in early May.  Over the first two weeks, we released one update, mainly to add a new feature and some bug fixes.  As we were preparing the second update, we thought it would be useful to take a look at the game’s performance, and the player behavior, to see if we could make any improvements that would increase user retention.

Stages

Stage 1: First introduction to the game, and the movement controls.  Players are given basic instructions and need to deliver two fares to complete the run.  If they deliver those fares, they are moved to the next phase.

Stage 2: First open-ended run.  Players deliver fares until they hit a performance criterion (based on a minimum number of fares delivered, and based on the number of stars they receive).  The first three deliveries are scripted, and intended to serve as a tutorial, and after that each destination is chosen randomly.  To advance, players must get 2 3-star deliveries and no 1-star deliveries.

Stage 3: Introducing other cars.  Players complete an open-ended run in larger chunk of the first town.  There is one other car wandering around in this area, and players need to avoid running into this car.  Criteria to advance are similar to Stage 2.

Stage 4: Full game in Galesburg.  All of the town of Galesburg is open, and as players deliver fares, more bystander and enemy cars are added to the town.  Power-ups and traps are also added. Criteria to advance are similar to Stage 2.

Stage 5: Crawfordsville is unlocked.  Players begin in Crawfordsville.  Otherwise, this phase is the same as Stage 4.

Player analytics

Taxi Dash is designed as a research study, and as part of that study we collect (anonymous) data about what people do during their gameplays.  Some of this data can be used simply as analytics: data about player behavior which might be useful for understanding what people like or don’t like about the game.  In mid-May (on Wednesday, May 15th) we looked at all of the data that had been submitted thus far.  According to the Amazon and Barnes and Noble developer consoles, we had less than 200 downloads of Taxi Dash on Amazon, but about 7,265 downloads from Barnes and Noble.  Of these ~7,500 downloads, we had analytic data recorded on our server from 3,451 installations.

Relase 4 stage distribution

The highest stage reached by each player in Taxi Dash. Most of the analytic data comes from players who only reached Stage 2 (or Stage 3), which are considered intro/tutorial stages.

Of the installations that we had analytics from, 1,878 came from the first update to Taxi Dash, and this is the version that we focused our attention on, as our interest was in how well this version was performing (in terms of engaging users). Our principle measure of engagement here was how far users progressed (through the five stages of training, see the Figure to the right). Overall, you can see that most players did not move past the introductory/tutorial levels (Stages 1-3). Most did not advance past Stage 2, and only about 2% (34 out of 1,878) made it to Stage 4 or higher.

Also, it appeared that very few participants only made it to Stage 1 (so it would seem that players are not quitting after the very first level), but these data may be misleading: recall that about half of the installations (at least) are not represented in these analytics. Based on how we collect analytics, it is likely that a player who did not persist past the first stage of training would not be represented in our data set (so, many of the “missing” installations may be players who did quit after the first stage of training).

Be that as it may, if at least half of the people who download our game are interested enough to try our second training level, we want to encourage them to experience more of the game (and to persist at least until Stage 4, where things get interesting!).

The number of times a player attempted Stage 2 versus the number of passengers delivered. Color indicates number of installations.

The number of times a player attempted Stage 2 versus the number of passengers delivered. Color indicates number of installations .

So, why don’t people make it to Stage 3, or Stage 4? One possibility is that they simply don’t like the game, and are quitting. If so, then we’ll likely need to make some significant improvements to the game to encourage those players to give us more of their time. But, another possibility is that they are giving the game a try, but becoming bored with the tutorial stages (namely Stage 2). This seemed like a reasonable guess when we started to look at the installations where people never moved past Stage 2. In those cases, we saw that many people were attempting Stage 2 multiple times, and they were delivering many passengers (sometimes in excess of 30 or 40).

Even player who only attempted Stage 2 one time (based on our analytics) often gave that Stage a great go. Shown in the plot to the left is the number of fared delivered by players who only tried Stage 2 once. As you can see in the figure below, most players in this group delivered more than 5 fares on their first try of Stage 2. And, across all of the players who did not move past Stage 2, 60% delivered at least 5 fares, and 20% made at least 4 attempts at Stage 2.

Number of fares delivered by players who only attempted Stage 2 one time.  Bars indicate number of installations.

Number of fares delivered by players who only attempted Stage 2 one time. Bars indicate number of installations.

So, what do we take away from these data? My first impression is that Stage 2 and 3 of training are now working right now, if players can attempt them multiple times and fail to progress to the next (and more interesting!) stages. I originally designed these stages (2 and 3) to protect players from becoming overwhelmed by the size of the city, and dealing with traps and enemy cars. But, the mechanisms to progress through the tutorial stages are too strict. If a player were to do well (to earn 3 stars for his or her deliveries), they should only have to deliver about 7 fares before they will clear the Stage. But, if a player regularly gets 1 star, or only infrequently gets 3 stars, they can stay in Stage 2 or Stage 3 permanently, and that seems to be happening to too many users.

The next iteration…

In the newest release of Taxi Dash, I’ve softened the requirements to progress through Stages 2 and 3. Now, the game keeps track of how many 3 star deliveries are made in each town. Once a player makes 3 deliveries in an attempt Stage 2, and has earned at least three 3 star deliveries (ever, including in Stage 1, or previous runs of Stage 2), they will move on to Stage 3. In Stage 3, there is a similar requirement. In either of these stages, if a player makes it to 10 deliveries in a single run, they will move on to the next stage, regardless of their peformance (the number of 3 star deliveries). And, once a player reaches Stage 4, new towns will be unlocked once they earn a total of ten 3 star deliveries in the previous town. Hopefully, we’ll see more people making it to the later stages in the game, and enjoying the game more for it!