[Python] Zero Metrics Integration
Repository
Summary
In modern development of games—regardless of console, PC, or mobile development—metrics have become an important tool for understanding and designing around player behavior. Although many resources exist to help integrate a metrics system into one's own product, many of these are designed for professional and commercial use, meaning a professional license which costs several thousands of dollars, or a contractual obligation. Neither of these is ideal for a student trying to learn the fundamentals of game programming and game design, so I looked for a more accessible service.
This led me to GameAnalytics.com, a site designed with the most minimal aspects of metrics but have some fundamental features which make it more desirable over other free alternatives:
Thanks to Jason Ericson who also developed around this service, and drove a lot of my design decisions when creating this for my teams at DigiPen.
This led me to GameAnalytics.com, a site designed with the most minimal aspects of metrics but have some fundamental features which make it more desirable over other free alternatives:
- No price
- Access to all data via HTTP POST requests using their REST API
- Customization and dashboard configuration using their web tool
- [DEPRECATED] Support for heatmaps which allow mapping in-game of metrics you care about.
Thanks to Jason Ericson who also developed around this service, and drove a lot of my design decisions when creating this for my teams at DigiPen.
Purpose
This implementation in python exists to work with DigiPen’s proprietary engine, the Zero Engine. The goal of this code is to work with Zero’s built in systems as well as the python library to give designers an easy interface to log and store events with their games. Much of this design was heavily inspired from my C++ interface, which was designed to use cURL and other exterior libraries that python has natively built in.
In lieu of using cURL for my web requests, Zero has a built in Web Requester component that allows for POST requests to be made in-engine. Unfortunately, this code is as of this writing blocking, which means that sending events is best done when not in crucial gameplay, such as transitioning between levels or when the player is otherwise loading or idle; players may see a small hang during this time.
[DEPRECATED]
Heatmap support can be done by using simple sprite primitives in zero, and by using the tuple of values created when calling the heatmap service from the GameAnalytics servers. The first value is the value associated with design events, and the second value is the point vector that was stored within each event.
In lieu of using cURL for my web requests, Zero has a built in Web Requester component that allows for POST requests to be made in-engine. Unfortunately, this code is as of this writing blocking, which means that sending events is best done when not in crucial gameplay, such as transitioning between levels or when the player is otherwise loading or idle; players may see a small hang during this time.
[DEPRECATED]
Heatmap support can be done by using simple sprite primitives in zero, and by using the tuple of values created when calling the heatmap service from the GameAnalytics servers. The first value is the value associated with design events, and the second value is the point vector that was stored within each event.
Metrics.py
'''-
File: Metrics.py A metrics system manager and definitions for the types of events available for use with the metrics system. Allof this functionality is implemented with the outlines given by GameAnalytics.com (http://www.gameanalytics.com) Support for the following features are implemented: --Sending events to a server --Obtaining heatmaps from a server __________ \____ /___________ ____ / // __ \_ __ \/ _ \ / /\ ___/| | \( <_> ) /_______ \___ >__| \____/ \/ \/ _____ __ .__ / \ _____/ |________|__| ____ ______ / \ / \_/ __ \ __\_ __ \ |/ ___\ / ___/ / Y \ ___/| | | | \/ \ \___ \___ \ \____|__ /\___ >__| |__| |__|\___ >____ > \/ \/ \/ \/ INFO Note: Requires Zero Build 2014/2/21 (8258) or later This plugin for Zero is standalone support for the services provided by http://www.GameAnalytics.com. The goal for this is to allow designers in GAT classes to be able to incorporate metrics into their projects, to better fuel design decisions and make stronger choices when crafting their game projects. README To use this service, first you will have to register with the gameanalytics service. To do so, follow the following steps: 0. If not already, please make sure you have Zero Build 8258 (February 2nd, 2014) or later installed on your computer. If you do not, the Metrics system will not work and raise an error. The 8258 build introduces expanded functionality in the WebRequester component native to Zero, which allows you to make HTTP POST requests necessary for access to GameAnalytics's servers. 1. Head to http://www.gameanalytics.com/ and click "Sign Up." From there complete their registration fields in order to register yourself with their website. You will need a valid email addres in order to use their services. You will create a studio for your projects here as well, and in this studio you can organize your games as well as give differing people access to studios (such as your GAM team at DigiPen). When you add your Game to the service, you will simply create a templated services page for your game which can then be used to display the events that you have sent to the server. 2. Once you have created an account, studio, and a game to your studio, we will need to pull the data from the analytics service to use it in conjunction with Zero. From your Home page, go to your game's "Settings" page, which can be accessed by clicking on the gear icon next to the title of your game on your studio. It can additionally be accessed from your game's webpage by clicking on "Game Settings" on the top right of the screen. Here you will find the three Game Keys that you will need in order to dispatch the Events in Zero. The different keys are: Game Key: This is the key that is connected to your game, and will be used in any sort of HTTP POST request with the server. Secret Key: This is an authorization key used to Data API Key: This is the key used in order to obtain data from the servers, such as Heatmaps and query information. 3. Add the python files "Metrics" and "TestScript" (optional) to your Zero Project. After these files are added, you will want to target a static object in your game that will be persistent. I suggest the Game object, selected via pressing Shift + G. Add the "Metrics" file to your Game object, as well as a WebRequester object. In the Metrics component, use the keys that has been obtained from your game's page to fill in the appropriate properties. The "TrackSessionLength" boolean (on by default) will automatically dispatch an event to your server when the game starts and when the game quits. This is important because GameAnalytics will track session length based off of the first and last events sent with the same session id. However, if you don't want this functionality you can turn off this by simply checking it. The "PrintStatements" boolean (off by default) will print out debug statements regarding dispatches to your console window. The dispatches are printed out in the following format: [Metrics:Dispatch] ### BEGIN DISPATCH ### [Metrics:Dispatch] <EVENT_ID> [Metrics:Dispatch] DATA: JSON [Metrics:Response] RESPONSE DATA: {"response":"value"} [Metrics:Response] RESPONSE CODE: 123 [Metrics:Dispatch] <...> [Metrics:Dispatch] ### END DISPATCH ### The "Build" value is a string value which can be used to denominate what types of builds your running tests on. This can be crucial if you're doing focus testing with a specific build of the game, or if crashes are occuring on a build that you know you have fixed on. NOTE: These can be any strings, not just numbers. 4. Now you're ready to start using your own calls to the metrics system for use. Use the "TestScript" file as a reference for proper ways to utilize the script, how to dispatch events, and how to generate heatmaps. CONTACT If you have problems, questions, ideas or suggestions, please contact me by emailing me at r.pannkuk@digipen.edu, or finding me in Edison. WEB SITE Visit the gameanalytics website in order to view your data and obtain the keys you need to use this service effectively: http://www.GameAnalytics.com NOTICE This data was made with permission from DigiPen Institute of Technology, and with the permission of the Zero Team under DigiPen. ''' import Zero import Events import Property import VectorMath import traceback import json import hashlib import uuid import datetime import time Events.MetricsRequestComplete = "MetricsRequestComplete"; Vec3 = VectorMath.Vector3; VecNull = Vec3(-9999, -9999, -9999); #Build version Build = "Unspecified"; #URL for the server URL = "http://api.gameanalytics.com/1/" #Generating ID for the user mac = hex(uuid.getnode()); m = hashlib.sha1(); m.update(mac.encode('utf-8')); UserID = m.hexdigest().upper(); #Unique session ID SessionID = uuid.uuid4(); #Possible categories of analytic events MetricCategories = Property.DeclareEnum("MetricCategories", ["design", "error", "business", "user"]); ''' Class: AnalyticsEvent This is the virtual class that will form the foundation for all of the events that you send out to the server. The class supports all of the properties that are common throughout all of the other event types which you will send out, whereas the derived classes will support the members that pertain only to them. ''' class AnalyticsEvent: EventID = Property.String(""); #Event Name Type = Property.Enum(enum=MetricCategories); #Type of event (DO NOT EDIT)-
def __init__(self, id): #Checking types of objects to make sure they are valid. if(not isinstance(id, str)): raise TypeException("id", id, str); self.EventID = id; pass; def Initialize(self, initializer): pass; def Write(self): #Writing out a dictionary that will be used when dumping with JSON #These are explicitly required by the analytics server #So do not edit them. message = {}; message["event_id"] = self.EventID; #ID to denote the event message["user_id"] = UserID; #UserID (Obtained from MAC address) message["session_id"] = str(SessionID); #Session ID (Always unique) message["build"] = str(Build); #Version of the game to test with return message; ''' Class: DesignEvent Parent: AnalyticsEvent These are the main event types used by game designers, where the user will provide a singular ID associated with the event, as well as a float value. All events that are not errors will be of type Design in this implementation, and includes things like "Game:Start", "BulletsFired", or anything else pertaining to your game. ''' class DesignEvent(AnalyticsEvent): Value = Property.Float(); #Value of the design event, something like number of bullets fired Area = Property.String(""); #Area the Event took place in Location = Property.Vector3(VecNull); #Location (exactly) of the event in the area-
def __init__(self, id, value, area, location): #Checking types of objects to make sure they are valid. if(not isinstance(id, str)): raise TypeException("id", id, str); if(not isinstance(value, float)): raise TypeException("value", value, float); if(not isinstance(area, str)): raise TypeException("area", area, str); if(not isinstance(location, Vec3)): raise TypeException("location", location, Vec3); self.EventID = id; #This is the ID the event will be referred to self.Value = value; #Value of the design event, something like number of bullets fired self.Area = area; #Area the Event took place in self.Location = location; #Location (exactly) of the event in the area self.Type = MetricCategories.design; pass; def Initialize(self, initializer): pass; def Write(self): #Using the parent class' JSON dump and adding Design Specific values message = AnalyticsEvent.Write(self); message["value"] = self.Value; message["area"] = self.Area; message["x"] = self.Location.x; message["y"] = self.Location.y; message["z"] = self.Location.z; return message; #DO NOT CHANGE THESE. These types are required by the analytics service. ErrorSeverity = Property.DeclareEnum("ErrorSeverity", ["Critical", "Error", "Warning", "Info", "Debug"]); ''' Class: ErrorEvent Parent: AnalyticsEvent This event type allows the user to send off a specific error message attached with their event. Typically this message is a raised exception, and a stack dump of the exception to track exactly where the bug can occur. These events can be easily found on GameAnalytics' Quality page or it's realtime page for currently occuring bugs, along with the stack dumps of these errors. ''' class ErrorEvent(AnalyticsEvent): Message = Property.String(); #Message to send with the error Severity = Property.Enum(enum=ErrorSeverity); #Severity of the error, using the above enum Area = Property.String(""); #Area the Event took place in Location = Property.Vector3(VecNull); #Location (exactly) of the event in the area-
def __init__(self, message, severity, area, location): #Checking types of objects to make sure they are valid. if(not isinstance(message, str)): raise TypeException("message", message, str); if(not severity in ErrorSeverity): raise Exception("Wrong Severity type. Please use Metrics.ErrorSeverity.<Critical, Error, Warning, Info, Debug>"); if(not isinstance(area, str)): raise TypeException("area", area, str); if(not isinstance(location, Vec3)): raise TypeException("location", location, vec3); self.Message = message; #Message to send with the error self.Severity = severity; #Severity of the error, using the above enum self.Area = area; #Area the Event took place in self.Location = location; #Location (exactly) of the event in the area self.Type = MetricCategories.error; pass; def Initialize(self, initializer): pass; def Write(self): #Using the parent class' JSON dump and adding Error Specific values message = AnalyticsEvent.Write(self) #Event_ID is not used in the error event class; instead messages are. del message["event_id"]; message["message"] = self.Message; message["severity"] = self.Severity.lower(); #Make sure severity is lower case message["area"] = self.Area; message["x"] = self.Location.x; message["y"] = self.Location.y; message["z"] = self.Location.z; return message; ''' Used for determining the type of web response that we were expecting after using the WebRequester component. Dispatch: Sending data to the server. Heatmap: Obtaining data from the server. ''' MetricsResponseType = Property.DeclareEnum("MetricsResponseType", ["Heatmap", "Dispatch"]); ''' class: Metrics requires: WebRequester This is the manager which parses and interprets all of the events added by the user, and then interacts using a WebRequester with the GameAnalytics server. This is where the bulk of functionality comes from, whereas the previous classes are simply containers of objects that are to be converted to JSON. ''' class Metrics: #Turn off (useful in cases where no internet connection, so no time out) Active = Property.Bool(True); #Keys, found in the project page GameKey = Property.String(""); SecretKey = Property.String(""); DataAPIKey = Property.String(""); #This determines whether or not you want automatic #tracking of session lengths in the game. TrackSessionLength = Property.Bool(True); #Turns on print statements to the console, which #can be useful for debugging events. PrintStatements = Property.Bool(False); ''' This is the current build of the game that you are on. Useful for things like "Alpha" builds or tracking versions. Especially useful in the case of tracking bugs between different versions. ''' Build = Property.String(Build); def Initialize(self, initializer):-
''' Takes the build property from the Metrics System and sets the global variable used in the Event classes. Additionally ensures that there is a unique SessionID upon init, so that game data can be tracked. ''' global Build, SessionID; Build = self.Build; SessionID = uuid.uuid4(); self.Messages = []; #Contains all of the messages to dispatch in one sitting self.Data = None; #Contains the data received from the Web Request self.ResponseType = None; #Flags the Response Type to expect from the Web Request #Checking for appropriate versions and components attached if(not self.Owner.WebRequester): raise Exception("Please attach a WebRequester component to the object which holds the Metrics component."); if(not self.Owner.WebRequester.SetHeader): raise Exception("Invalid version of Zero. Please update to a version that has required functionality."); #Connecting to the WebResponse event to obtain our data Zero.Connect(self.Owner, Events.WebResponse, self.OnWebResponse); ''' Setting up the initial Game:Start and Game:End events This will ensure that we can get a valid duration for game length. The same approach should be taken with "Level:Start" and "Level:End" ones. ''' if(self.TrackSessionLength): self.GameStart(); pass ''' Function: GameStart This is used when tracking session length to determine the total session length of the game played. The way that GameAnalytics parses session length is by taking the first instance of an event with a unique session ID, and the last instace of an event with a unique session ID, and finding the time differential between the two. ''' def GameStart(self):-
# Checking for Active falg if(not self.Active): return; ''' The way that session lengths are measured on GameAnalytics is to measure the time from the first event sent, to the last event sent. With this event we are ensuring an event is sent immediatley upon game start, so that we can begin tracking session length. ''' self.AddEvent( DesignEvent("Game:Start", #Event ID 1.0, #Value "NULL", #Level (not applicable) VecNull) #Position (not applicable) ); self.Dispatch(); pass ''' Function: Destroyed This is used when tracking session length to determine the total session length of the game played. The way that GameAnalytics parses session length is by taking the first instance of an event with a unique session ID, and the last instace of an event with a unique session ID, and finding the time differential between the two. ''' def Destroyed(self):-
#Checking for Active flag if(not self.Active): return; #Only enabled when tracking session length if(self.TrackSessionLength): ''' The way that session lengths are measured on GameAnalytics is to measure the time from the first event sent, to the last event sent. With this event we are ensuring an event is sent immediatley upon game start, so that we can begin tracking session length. ''' self.AddEvent( DesignEvent("Game:End", #Event ID 1.0, #Value "NULL", #Level (not applicable) VecNull) #Position (not applicable) ); self.Dispatch(); pass ''' Function: OnWebResponse This is an event handler for the WebRequester, which will allow us to access the data the POST request gives us after the request has been made. The self.ResponseType enum will dictate whether to take additional steps required for generating the points on a heatmap. It will print to the console the WebResponse Data as well as the Response Code for the data. Good Signals: Response Code: 200 Data: {"status":"ok"} (For Dispatching Events) Data: {"x": [...], "y": [...], "z": [...]} (For Obtaining Heatmaps) Bad Signals: Response Code: 0 (This likely means your network connection isn't working as intended) Response Code: 401 (Unauthorized Access, this likely means that your keys are wrong) Response Code: 404 (Invalid URL, if you got this you borked the URL somehow. Good luck.) ''' def OnWebResponse(self, WebResponseEvent):-
#Checking for Active Flag if(not self.Active): return; if(self.PrintStatements): #Printing out the data to the consoles print("[Metrics:Response] RESPONSE DATA: " + str(WebResponseEvent.Data)); print("[Metrics:Response] RESPONSE CODE: " + str(WebResponseEvent.ResponseCode)); #Generating an event to be sent out. e = Zero.ScriptEvent(); e.Data = WebResponseEvent.Data; e.Response = WebResponseEvent.ResponseCode; #Check that we actually made it to the server if(WebResponseEvent.ResponseCode == 200): #This branch is completed when attempting Heatmap polling. if(self.ResponseType == MetricsResponseType.Heatmap): ''' The data retrieved is in JSON format, which means we can use Python's native JSON library to reinterpret the data into a Python list. ''' dict = json.loads(WebResponseEvent.Data); self.Data = []; #Error check for a valid dictionary returned if("error" in dict): print("[Metrics:Response]********ERROR********"); else: ''' This takes each value in the JSON object and splits the array up into individual tuples, the first part being the position of the event, and the second part being the count (number of hits) of that event. ''' for i in range(len(dict["x"])): self.Data.append((Vec3(dict["x"][i], dict["y"][i], dict["z"][i]), dict["value"][i])); ''' The data comes in a collection of arrays for the following values: x: The x component of a point on the heatmap y: The y component of a point on the heatmap z: The z component of a point on the heatmap value: ***THIS IS NOT THE VALUE YOU SENT THE EVENT WITH*** This it the count for the number of points found will the same values. Many points are generated in the heatmap, and take up the following form: { "x": [1,2,3,...,n], "y": [1,2,3,...,n], "z": [1,2,3,...,n], "value": [1,1,1,...,n], } Note that the count is the same for each variable, because each event sent is guaranteed to have all four values. ''' e.Data = self.Data; ''' This is the event that is sent out when the Web Request has finished and received it's POST data. As of build 8258, Zero does not separate a thread for it's WebRequester, so it will work to simply access Metrics.Data directly after your Dispatch or Request is made, but this is NOT the ideal practice. It is safter to use the Event, in the instance that Zero changes in the futureto support threaded POST requests. ''' self.Owner.DispatchEvent(Events.MetricsRequestComplete, e); pass ''' Function: AddEvent This helper function simply adds a user generated event to the list of events to be sent out when calling self.Dispatch. NOTE: Your event will not be sent until you Dispatch the event. ''' def AddEvent(self, Event):-
#Checking for Active Flag if(not self.Active): return; #Checking for the proper type. Note that AnalyticsEvent is NOT supported. if(not (isinstance(Event, DesignEvent) or isinstance(Event, ErrorEvent))): raise TypeException("Event", Event, (DesignEvent, ErrorEvent)); #Appending the type of the message, and the dictionary of event items self.Messages.append((Event.Type, Event.Write())); pass; ''' Function: Dispatch Takes any events that were stored in the self.Messages list and then dispatches them to the server one by one. I use a for loop to do this, which is slightly inefficient but easier to understand and debug than a JSON array of messages. ''' def Dispatch(self):-
#Checking for Active Flag if(not self.Active): return; #Debug Printing if(self.PrintStatements): print("[Metrics:Dispatch] ### BEGIN DISPATCH ###"); #Looping through all of the messages in the log for message in self.Messages: #Debug Printing if(self.PrintStatements): if(message[0] == MetricCategories.design): print("[Metrics:Dispatch] <" + message[1]["event_id"] + ">"); elif(message[0] == MetricCategories.error): print("[Metrics:Dispatch] <" + message[1]["message"] + ">"); print("[Metrics:Dispatch] DATA: " + str(message[1])); #The first part of the tuple is the category of the message <design, error> category = message[0]; #The second part of the tuple is the values of the message in a dict object json_message = json.dumps(message[1]); #Using the hash library and md5 as per the instructions from http://support.gameanalytics.com/hc/en-us/articles/200841786-Python digest = hashlib.md5() digest.update(json_message.encode('utf-8')) digest.update(self.SecretKey.encode('utf-8')) #Set the response type for checking. #Dispatch implies we are sending data out to the server. self.ResponseType = MetricsResponseType.Dispatch; #Sending the request WebRequester = self.Owner.WebRequester; WebRequester.Clear(); url = "%s/%s/%s" % (URL, self.GameKey, category) WebRequester.Url = url; WebRequester.SetHeader('Authorization', digest.hexdigest()); WebRequester.SetPostData(json_message); WebRequester.Run(); self.Messages = []; #Debug Printing if(self.PrintStatements): print("[Metrics:Dispatch] ### END DISPATCH ###"); pass; ''' Function: RequestData Used to obtain heatmap data from an event or tuple of events so that it may be used in whatever way desirable by the user. ''' def RequestData(self, event_type, #Type of the event <design, error> event_ids, #ID's of the events to obtain area, #Level Name or Area build="", #(Optional) Build Version startDate=datetime.date(datetime.MINYEAR, 1, 1), #(Optional) Start Time endDate=datetime.datetime.now()): #(Optional) End Time-
print(build); #Checking for Active Flag if(not self.Active): return; ''' This entire block is simply a means of doing error checking based on user input. Each line is simply a check to make sure the type is correct (or in some cases the actual values are correct, such as in event_type) to make usability as straightforward as possible. This isn't necessary but I think it's a nice way to catch the user's mistakes. ''' if(not (event_type == "design" or event_type == "error")): raise Exception("Improper value for var event_type" + str(event_type) + ". Should be: <\"design\", \"error\">"); if(isinstance(event_ids, (list, tuple))): events_string = ""; for event_id in event_ids: events_string += str(event_id) + "|"; events_string = events_string[:-1]; elif(isinstance(event_ids, str)): events_string = event_ids; else: raise TypeException("event_ids", event_ids, (str, (str, str))); if(not isinstance(area, str)): raise TypeException("area", area, str); if(isinstance(startDate, datetime.date)): startDate = time.mktime(startDate.timetuple()); elif(not isinstance(startDate, float)): raise TypeException("startDate", startDate, (datetime.date, float)); if(isinstance(endDate, datetime.date)): endDate = time.mktime(endDate.timetuple()); elif(not isinstance(endDate, float)): raise TypeException("endDate", endDate, (datetime.date, float)); if(not isinstance(build, str)): raise TypeException("build", build, str); #Info for the request, pertaining to the attributes given in the function call requestInfo = ""; requestInfo += "game_key=" + self.GameKey; requestInfo += "&event_ids=" + events_string; requestInfo += "&area=" + area; requestInfo += "&start_ts=" + str(startDate); requestInfo += "&end_ts=" + str(endDate); #Check if there was a build requested if(build != ""): requestInfo += "&build=" + build; #URL to fetch the data from url = "http://data-api.gameanalytics.com/heatmap/?" + requestInfo; #Create the authorization header as per instructions from gameanalytics.com header = requestInfo + self.DataAPIKey; digest = hashlib.md5(); digest.update(header.encode('utf-8')); #Set the response type for checking #Heatmap implies we are obtaining data to store as a heatmap self.ResponseType = MetricsResponseType.Heatmap; #Debug Printing if(self.PrintStatements): print("[Metrics:Heatmaps] <" + events_string + ">"); print("[Metrics:Heatmaps] URL: " + url); #Run the web requester to fetch the data WebRequester = self.Owner.WebRequester; WebRequester.Clear(); WebRequester.Url = url; WebRequester.SetHeader("Authorization", digest.hexdigest()); WebRequester.SetPostData(url); WebRequester.Run(); pass; ''' Class: TypeException parent: Exception This is an exception class that takes a variable and a proper type. Should be used when someone does not use the proper type and instead should be using another type of object. ''' class TypeException(Exception): def __init__(self, string, variable, desired_types):-
msg = ""; msg += "Improper type for \"" + string + "\" (" + variable.__class__.__name__ +"). Should be: "; #Check for tuple if(isinstance(desired_types, tuple)): #Add all types to the string for single_type in desired_types: msg += str(single_type); msg += ", "; #Strip last comma msg = msg[:-1]; #Default case else: msg += str(desired_types); #Send to Exception Initializer Exception.__init__(self, msg); Zero.RegisterComponent("Metrics", Metrics)
