GameSound, Quantitative Games Analysis, and the Digital Humanities

This article relates to the 2018 CSDH/SCHN conference proceedings. This paper outlines Michael Iantorno’s and Melissa Mony’s experiences with quantitative game analysis by summarizing the first year of development of the prototype ludomusicological database GameSound. To further the discussion, this article also summarizes and analyzes the work of fellow digital humanities scholar Jason Bradshaw, who applied intriguing types of tool-based analysis to BioShock Infinite. To conclude, the paper hypothesizes where this type of research could lead in the future: both for GameSound and for other projects using similar methods and methodologies. Resume Cet article presente les experiences de Michael Iantorno et de Melissa Mony faites avec des analyses de jeu quantitatives, en resumant la premiere annee de developpement de la base de donnees prototype ludomusicale GameSound. Pour approfondir la discussion, cet article resume et analyse egalement l’oeuvre de Jason Bradshaw et de Dr. Adrienne Shaw, qui emploient des types intrigants d’analyses de jeu quantitatives et qualitatives dans leurs propres projets, respectivement la BioShock Infinite and Feminist Theory : A Technical Approach et The LGBTQ Video Game Archive. Pour conclure, cet article formule une hypothese concernant l’avenir de ce genre de recherche : non seulement pour GameSound mais aussi pour d’autres projets qui se servent de methodes et de methodologies similaires. Mots-cles: etudes de jeux; la ludomusicologie; humanites numeriques; recherche quantitative; bases de donnees

Cet article présente les expériences de Michael Iantorno et de Melissa Mony faites avec des analyses de jeu quantitatives, en résumant la première année de développement de la base de données prototype ludomusicale GameSound. Pour approfondir la discussion, cet article résume et analyse également l'oeuvre de Jason Bradshaw et de Dr. Adrienne Shaw, qui emploient des types intrigants d'analyses de jeu quantitatives et qualitatives dans leurs propres projets, respectivement la BioShock Infinite and Feminist Theory : A Technical Approach et The LGBTQ Video Game Archive. Pour conclure, cet article formule une hypothèse concernant l'avenir de ce genre de recherche : non seulement pour GameSound mais aussi pour d'autres projets qui se servent de méthodes et de méthodologies similaires.

Introduction
For digital humanities scholars, breaking down a videogame into its component parts may seem like an obvious strategy for better understanding it. Many of these researchers are players themselves, after all, and the act of play implicitly invites this sort of systematic deconstruction.
As a player engages with a videogame, they accumulate rules knowledge, acquire in-game resources, and develop a tacit understanding of what the game asks of them, slowly inching toward a deepened comprehension of the entire play experience.
Only through this accumulation of resources and expertise can they improve their performance within a videogame, eventually completing objectives and perhaps even beating a title in its entirety.
From an academic perspective, this deconstructive learning process serves a purpose other than game mastery. Instead of forging a better understanding of a game through play (as intended by the developer), technically-minded scholars can separate a videogame into its component parts, which can then be arranged and rearranged to facilitate the needs of their research. This process can be as simple as using save files or cheat codes to access parts of a game strategically, a process that often manifests as targeted or repeated playthroughs, or can involve delving into a game's code and assets in an attempt to peek behind the curtains-scrutinizing the numerous individual elements that make up a videogame title. It is this latter form of analysis that we are interested in exploring within this paper. By accessing game assets directly, rather than solely through play, we believe that scholars have the opportunity to analyze videogames in new ways-not just as a series of set pieces or vignettes pre-determined by the original developer. When broken into parts and viewed as a collection of diverse assets, rather than a homogenous whole, new angles of research become possible through the adoption of both established and emerging forms of quantitative analysis. This paper discusses one potential avenue for this type of videogame analysis by documenting the creation, functionality, and potential applications of GameSound, Iantorno: GameSound, Quantitative Games Analysis, and the Digital Humanities Art. 2, page 3 of 18 a digital humanities database project developed between 2017 and 2018. 1 We developed GameSound as a prototype ludomusicological database, with the intent of providing users with easy access to the music and sound effects present within videogames. By making these audio files accessible through a web-based interface, and supplementing them with technical and contextual data, our hope is that GameSound could be used to facilitate new types of academic research. Using Civilization IV as a case study and HEURIST to build the database, GameSound currently provides access to over 2000 music and sound effect files. This paper begins with a brief overview of quantitative game analysis, both as it is defined for this particular digital humanities project and how it has been used in other projects, specifically Jason Bradshaw's "BioShock Infinite and Feminist Theory: A Technical Approach." We then discuss the functionality of the database itself, while also outlining the technical, legal, and theoretical challenges that arose while designing it. Finally, we hypothesize where this research could lead in the future: both for GameSound and other similar projects.
However, we would be remiss to begin these discussions without first touching on the academic field that stands behind GameSound, ludomusicology. An emerging sub-discipline of musicology, ludomusicology focuses on the academic study of the audio present in videogames. Primarily concerned with the direct study of a videogame's music and sound effects, ludomusicology also interrogates how we study audio within the context of digital software. Since the neologism was coined in 2007, the field has expanded to include music games, fan cultural music practices, live concerts, and the impact that game music has had on other musical genres tension between art and entertainment that arises when videogames remix classical music (Gibbons 2016). Like many topics associated with the digital humanities, ludomusicology traverses disciplines: fostering collaborations with computer science, film and media studies, and communications. GameSound is our first foray into this type of research, originally conceived as a digital humanities class assignment at McGill University, and we believe the database has potential academic, professional, and hobbyist applications.

Quantitative videogame analysis
We describe GameSound as a tool for enabling new types of quantitative videogame analysis, but we also acknowledge that the term "quantitative" can be vague and requires further elaboration. In the context of GameSound, it reflects an approach that is comprehensive (collecting all of the audio in a given game), measurable (defining collected game audio using quantities and simple identifiers), and statistical (enabling the comparison of data using tables, charts, faceted searches, and visualizations). In contrast to qualitative videogame analysis methods, which commonly rely on written logs constructed through repeated playthroughs (Consalvo and Dutton 2006), quantitative analysis is much more tool-oriented and focuses on parameters that can be measured or counted. Although a somewhat new approach, quantitative videogame analysis can be facilitated by adopting existing digital humanities tools and established methods that have their origins in fields such as literature, history, and philosophy. The practice of isolating and extracting specific words and phrases has enabled new types of distant reading in literature, for example, and many of the methods we use to analyze books can be co-opted for analyzing various aspects of a videogame, from text to media assets. of digital analysis can also lend themselves to mediums born of the digital age" (Bradshaw 2018). After acquiring the game's script, Bradshaw fed the entirety of the text into Voyant Tools-a popular piece of textual analysis software-to determine the frequency of certain words and where they occurred in the game's timeline.
Bradshaw tracked how Elizabeth was referred to throughout the game as well as how she was treated by other characters, resulting in a measured, textual character arc. By paying keen attention to labels such as "baby" or "child," he analyzed her function in the storyline and how it could be "an allusion to the treatment of women in contemporary society" (Bradshaw 2018). Although by no means an authoritative take on the game's narrative, the study demonstrates a progressive methodological approach for game studies: one that invites complementary quantitative analysis using methods generally reserved for literary works.
Much like Bradshaw's work, GameSound also takes a deconstructive approach, but focuses on the sound within a videogame rather the text. Sound was chosen as the focus for the project as we felt that it was one of the more difficult assets of a videogame to isolate-often framed as a secondary element to mechanics, visuals, or narrative-and because of our own experiences in professional sound and music theory.
By putting an emphasis on searchability and utilizing a variety of both technical and ludological descriptors, GameSound aims to facilitate access to a videogame's audio elements without having to rely on playthroughs or secondary resources. Once the heavy lifting of data input and formatting is complete, GameSound users have the freedom to access audio through faceted searches, create custom reports, and even embed content on the web for easy collaboration and knowledge dissemination. This approach allows for a rapid oscillation between different types of analysis, some of which may not have otherwise been available. We have outlined three speculative use cases for the database listed below:

Constructing and using GameSound
GameSound was created using HEURIST, a free platform for scholars in the digital humanities that enables online database construction, mixed-media assets, and dynamic data visualization. Like many digital humanities tools, HEURIST pursues a certain level of accessibility for its user base: HEURIST's research-driven data management system puts the user in charge, allowing them to design, create, manage, analyse and publish their own richly-structured database(s) within hours, through a simple web interface, without the need for programmers or consultants. (Sydney University 2018) After some experimentation with other tools, we chose HEURIST for this project primarily due to its ease-of-use. While we may be studying a subject that is steeped in code, we are certainly not experienced computer engineers or web developers (nor did we want to place this expectation on our collaborators). Thus, it was necessary to find a database tool that would take care of most of the heavy lifting for us while providing a very shallow learning curve for additional participants. In a way, this embodies one of the key design sensibilities for GameSound: a desire to create a useful resource that, at the same time, is easily accessible for game scholars, ludomusicologists, and independent researchers.
The ultimate goal for the database is to include videogames from different platforms and eras, but the initial dataset focuses entirely on the sounds found  Through the use of HEURIST's back-end tools, GameSound users are able to export customized reports that contain entries determined by parameters that exist within the database. Seen in Figure 3 are two excerpts from an enormous report that contains every single GameSound entry that possesses both a screenshot and a video link. Reports are quite flexible in both content and layout-as this report is embedded on GameSound's homepage, it has been customized to mimic the font styles from the website's CSS files. Able to query any type of data that is present in the database, reports are a versatile way to share targeted sets of data with other researchers.
In addition to the database itself, GameSound also exists as a web resource for those who are interested in ludomusicology and quantitative game analysis. The website currently hosts documentation outlining the project's methods and explains how many aspects of the research were conceived. Similar to how the database provides transparent access to videogame audio, we hope that the website provides insight into how GameSound was developed.

Challenges in designing the database
Ludomusicology can be a complicated endeavour for researchers, scholars, and digital humanists. Conventional qualitative methodologies prevalent in ludomusicology, such as analytical play (Summers and Hannigan 2016, 52), can lack the capacity to adequately access and isolate audio or investigate the role of interactivity-the layers "between the operations of a machine and the instructions given to it by an operator" (Burdick 2012, 53). Commercial soundtracks and other official releases of game audio can be somewhat unreliable sources, as composers may alter the recordings from their original presentation (while completely divorcing them from their in-game context).
Thus, one of the key challenges in creating a ludomusicological database is gaining access to videogame audio directly without dismissing its role within gameplay.
This challenge is exacerbated by the opacity of videogame file structures-much of a game's audio assets are obfuscated through layers of code, file compression, and technical protection measures. There is a notable lack of academic tools that can penetrate these layers, and most of the existing software used to extract audio and game code falls squarely into the realm of modding or hobbyism. Thus, researchers must seek out games with open file structures (as we did with Civilization IV ) or use independently developed tools that have very little documentation or support. In addition to the technical challenges of acquiring audio and determining its purpose within the framework of the game, one of the biggest hurdles faced when designing GameSound's initial prototype was deciding which data types we would include in the database. As GameSound was not created with a specific research project in mind-focusing more on potential applications-much of the initial data gathering was speculative in nature. As a result, we experimented with an extremely broad range of technical and ludomusicological data throughout the database's development. This, admittedly, may have led to some arbitrary decisions regarding data types, but gave us permission to contemplate an enormous variety of potential applications for the database. Code snippets and written descriptors were both strongly considered during the prototyping process before we whittled down the selection to its current state, which focuses more on measurable technical parameters, game media, and ludomusicological identifiers. These data types were selected for practical reasons, as much of their extraction could be automated, as well as their perceived usefulness for digital humanities scholars.
In an effort to better place GameSound within current ludomusicological discourses, instead of developing our own identification system we decided to adopt the IEZA framework-a two-dimensional method for describing sound in computer games. Designed by Sander Huiberts and Richard van Tol at the Utrecht School of the Arts, the IEZA framework (Figure 4) provides an effective vocabulary for audio classification. The vertical dimension in the framework makes a distinction between audio originating from inside the fictional game world (diegetic), such as the footsteps of a game character, and sound coming from outside the fictional game world (non-diegetic), such as the title's musical score. • Effect: sounds rooted in the game world that are triggered directly by player action, such as dialogue, footsteps, and gunshots.
• Zone: sounds rooted in the game world that are not directly triggered by player action, such as rain, wind, or city noise.
• Interface: sounds that exist outside the game's fictional setting that are triggered directly by player action, such as beeps and clicks emitted by a game's menu or HUD.
• Affect: sounds that exist outside the game's fictional setting that are not triggered directly by player action, such as a game's musical score or ominous drones in a horror game.
These categorizations help to establish the importance of interactivity in videogame audio-the simple fact that "the body cannot be removed from the experience of videogame play" (Collins 2013, 3). Players are not just the receiver of a sound signalas with radio, television, and film-but also the transmitter. In-game actions may trigger dialogue, sound effects, music, and ambient sounds directly (by clicking a button in the interface) or indirectly (through timed events or algorithms). Thus,

Conclusion
An obvious question for GameSound is: "what are the next steps for the project?" the obvious path forward. We are currently canvassing the game studies community to seek out collaborations with those who may find GamesSound's unique toolset useful for their own research projects. This expansion is important as, although the development process has been enlightening, a digital humanities project such as this cannot be truly evaluated until it has moved beyond speculative use and has been tested in multiple academic research projects.
As with the prototype, any potential collaboration would likely revolve around the addition of a single videogame. Much like we did with Civilization IV, database updates will focus around the acquisition of audio assets from a game, followed by a sorting process (based around the researcher's needs as well as the existing categories present in the database). This fresh infusion of data will offer new opportunities for introspection and revision, and the database may be altered or completely rebuilt to include new categories, data types, or search functionality (such as the ability to quickly navigate between titles). Essentially, our goal would be to allow new collaborators to sift through the videogame of their choice with ease, providing them with access to the audio elements of a game without relying on repeated playthroughs or unreliable secondary sources. As with any addition to the databasewhich, barring copyright concerns, will always be publicly available-additional data will also provide scholars from all across the game studies and digital humanities communities with an opportunity to reflect upon the technical parameters of game audio and to experiment with quantitative game analysis tools and applications. As the database moves toward a multitude of videogames, rather than a single one, we will be presented with further opportunities to measure its value as an ongoing digital humanities project.
GameSound is an iterative work. Beyond the research possibilities that it provides, it is meant to explore the ongoing technical challenges ludomusicologists face, such as data accessibility, intellectual property concerns, and the lack of established standards. Over time, we hope to collaborate with scholars across the world to expand both the scope and functionality of the database, while constantly interrogating its efficacy as a research tool. Just as we hope to learn more about videogames by breaking them down into their component parts, it is our belief