What defactoring as a method helps us understand

There is a tendency both in scholars and engineers to separate things (Bowker and Star 1999). We can see one such separation in one of the largest communities of digital praxis in the humanities, the TEI-XML movement. Inspired by early developments in electronic typesetting (Goldfarb 1996) both textual scholars and engineers arrived upon the idea of separation of form and content (DeRose et al. 1990): there is the textual information (“Nixon resigns”), and there is how that information looks (e.g. bold large caps in the case of a newspaper heading). Thus, in TEI-XML an unproblematic separation of information and layout is assumed. On closer inspection however, such a separation is not as unproblematic at all (Welch 2010, Galey 2010). Form demands to be part of meaning and interpretation as is dramatically clear from looking at just one poem by William Blake. Yet such separation has emerged in science and research: data tends to be separated from research as an analytical process, and the creation of digital research objects (such as digital data and analytic code) goes often unrecognized as intellectual research work and is considered ‘mere’ supportive material labor (Burgess and Hamming 2011). Data is mostly regarded as a neutral and research independent entity, indeed something ‘given’ as the Latin root suggests. That the state of data are not quite as straightforward has been argued before (Galey 2010, Drucker 2011, and Gitelman 2013). From our defactoring experience we derive the same finding; there are rich stories to tell about the interactions between code and data.

Code can be read and reviewed independent of its context and purpose, as a static textual object. In such a case one looks critically at the structure of the code (are separate steps of the process clearly delineated and pushed to individual subroutines to create a clearly articulated and maintainable process, are there considerable performance issues, have existing proven libraries been used?) This kind of criticism—we could compare it to textual criticism—is informative but in a way that is wholly unconnected to the context of its execution. It is like picking apart every single cog in a mechanical clock to judge if it is well built, but without being interested in what context and for what purpose it will tell time. This would be code review as practiced in an industrial setting. Code review takes the structural and technical quality of code into consideration only insofar that obvious analytical errors should be pointed out, judged against measures of performance and industrial scalability and maintainability. However, this approach has little relevance for the bespoke code of scholarly productions (level four in Hinsen’s model of scientific software), it is relatively “okay for academic code to suck” as compared to the best practices of industrial code-work. But what about best practices for the bespoke code of scholarly research? What about code that “only runs once” and whose purpose is insight rather than staying around as reusable software? We put forth defactoring as a technique for unveiling the workings of bespoke code to the scholarly reader (and potentially a larger audience). We cast it as a form of close reading that draws out the interaction between code and the content, the data, and the subject matter. Essentially, we think, defactoring is a technique to read and critique those moments of interaction. It is there that data, analytic code, and subject matter co-exist in a meaningful dynamic. It is at these points that defactoring affords a scholar to ask—not unlike she would while enacting literary criticism: what happens here and what does that mean? Whereas the original code is mere supportive materials, the defactored and critically read code morphs into a first-order computational narrative that elevates the documentary residue of the computational analysis: a critical interpretation of the code

In this sense we found that defactoring is more than just a method to open up bespoke code to close reading in the (digital) humanities. It also shows how code literacy, just as “conventional” literacy, affords an interpretative intellectual engagement with the work of other scholars, which is interesting in itself. In addition, we think that the potential of defactoring reaches beyond the domain of digital humanities. As a critical method it intersects with the domains of Critical Code Studies and Cultural Analytics as well, and could as a matter of fact, prove viable and useful in any scientific domain where bespoke code is used or studied

On an epistemological level once again it appears that we cannot carve up research in neatly containerized independent activities of which the individual quality can be easily established and aggregated to a sum that is greater than the parts. The ‘greater’ is exactly in the relations that exist between the various activities and that become opaque if the focus is put on what is inside the different containers. This is why we would push even farther than saying that data and code are not unproblematically separable entities. Indeed, we would argue that they are both intrinsic parts of a grander story that Underwood and Sellers tell us and which consists of several intertwined narratives: there is a narrative that is told by the code, one that is told by the comments we found in that code, there is a narrative of data transformations. These narratives together become the premises of an overarching narrative that results first as a Figshare contribution, and later as an MLQ publication. These narratives are all stacked turtles, and they all deserve proper introspection.

Quite logically with each stacked narrative the contribution of each underlying narrative becomes a little more opaque. The MLQ article suggests an unproblematic, neat, and polished process of question-data-analysis-result. But it is only thanks to their open science like attitude that Underwood and Sellers grant us insight into the very process that led from data mongering and structuring through several iterations of analysis to a presentable result. Underwood and Sellers went through several iterations of refining their story. The Figshare contribution and the code give us much insight in what the real research looked like for which the MLQ article, in agreement with Jon Claerbout’s ideas (Buckheit and Donoho 1995), turns out to be a mere advertising of the underlying scholarship. In contrast to what we argue here—that data and code deserve more exposure and evaluation as being integral parts of a research narrative—we observed in the succession of narrative transformations that the aspect and contribution of code became not only more opaque with every stacked narrative, but vanished altogether from the MLQ article. This gap is not a fault of Underwood and Sellers, but rather deeply embedded in the practices and expectations of scholarly publishing.