It seems that every time I leave town to visit family, some issue comes up with my research games that is difficult to deal with while travelling. This Christmas holiday was no exception, as I noticed on Christmas day that I had an email with the subject heading “URGENT: Account Suspension.” The automated message was not encouraging:
System administration was forced to suspend your site in an emergency to prevent server and system overloads. We should have more information forthcoming and will attempt to reach out to you shortly…
Scary stuff, but when I attempted to load the site, it seemed to be running fine, and I did not see any unusual activity (though Resource Usage graph provided by my webhost showed a spike in activity). Since the site appeared to have not been suspended, I assumed there were no major issues, but I did sent a reply to the tech support at my webhost. I received a reply fairly quickly, and it turned out my most recent research game, Taxi Dash, was generating a lot of POST requests to a page on my site (in excess of 70k per day at the worst).
Eventually, my site was suspended (twice) as the resource use triggered automated suspensions. The tech support staff in general were very helpful, and although they did want to move me up to a more expensive plan, they did reactivate my site, and have given me some time to get the resource use under control. For behaviorgames.com, I use a shared hosting package, which allows me to run the site for a modest cost. The shared hosting package I use has limits, though, in how much of the server CPU it should be using in a day. Usually this CPU cap isn’t a problem, since the research games that I am working with are distributed mostly through mobile app stores (on iOS and Android), and I use the website primarily to provide information about the games and to collect data. However, the version of Taxi Dash that was live on two stores (Barnes and Noble, and Amazon), was set up to communicate with the server too often, it turns out. Each installation sends high scores, information on how much of the game a player has completed, and more detailed analytics that help us test our research hypotheses, and improve the game.
And, in the way that I had structured these requests, each installation was generating a LOT of POST requests. Especially around Christmas, when the number of installations from the Barnes and Noble app store shot up dramatically. I had expected a lot of new installs, as there is typically a holiday season bump around Christmas in the U.S., but in the past this has not caused problems with my website. The difference this year was that as new users downloaded Taxi Dash, the demand on my site was increasing dramatically for each installation.
Even now, after the Christmas rush has died down, I am still seeing a lot of traffic generated by the game. I have had only 7,581 unique visitors in the first 6 days of January, but those visitors have generated 289k page requests. Each of those requests attempts to connect to the mySQL database that I use for saving data, and that combination of traffic with the processing required to connect to the database seems to be the main issue.
For now, I have disabled the script that handles communicating with Taxi Dash, and I hope this will be enough to keep the site from being suspended again. My overall CPU usage is down in the past few days (as you can see in the graph above), but it’s difficult for me to say if that is from a change in the script, or if it is just due to a decrease in traffic to the page (though, I am still seeing 50-75k page requests in most days so far in January, this is still lower than I saw at the peak around Christmas (of 80-95k)). In any case, a new version of Taxi Dash has been submitted to both Barnes and Noble and Amazon, and I hope they will both go live this week. In the meantime, now I can get back to doing some research and planning for courses for next semester!