Pregnancy, space discovery, financial institutions, electronic music, development aid, international migration and zoo design are rarely, if ever, discussed at the same workshop. What connects them is the fact that they all have been subjected to testing in society. The tests that were examined during the international workshop Put to the Test: Critical Evaluations of Testing which took place at Warwick in London last December, took various forms: from a small and inexpensive over-the counter device, to large-scale laboratory-like arrangements, and complex computational modelling. What they shared was their location, as the workshop aimed to investigate forms of testing “beyond the laboratory”, in social environments, from a sociological perspective. The proliferation of tests in today’s societies is clearly noticeable. We constantly test and are being tested, both in the university and in everyday situations. Invented as relatively formalized procedures for determining the properties of partially unknown entities, tests escaped the ostensibly regulated realm of science quite some time ago. Their ubiquity in social life demands asking critical questions about different roles they have in society, how and by whom they are curated, and what are their justification, internal operations and multiple consequences.
The workshop, organised by Noortje Marres (University of Warwick) and David Stark (University of Warwick/Columbia University), aimed to engage specifically with the social consequences of testing, its organizational, creative and political dimensions and to ascertain the degree to which testing is amendable to critical analysis. In the words of one of the organizers, the event was an attempt to “test the test”. A rich selection of analytical perspectives from sociology, science & technology studies, media studies and design research helped to surface common themes and effects elicited by testing in society. It was recognized from the outset that it would be extremely difficult to construct a traditional, formal, typology of tests based for instance on methodology, status of the subject-matter, temporality, and/or efficacy of tests. Instead, the workshop’s investigations were most fruitful when they unearthed the capacity of tests to modify actors’ relations, capacities and contexts, connections between different levels of social reality, or tensions between unsettling and confirming capacities of tests, in short, where they managed to specify the consequences of the conduct of experiments, trials and try-outs in society.
A few common strands surfaced in the presentations and discussions during the workshop. A major one is the issue of ambiguity, understood as a precondition for testing and a source of its creative possibilities as well as one of its main outcomes. In line with experimental scientific traditions that are also present in organizational and political change management, many forms of testing aim at reduction of ambiguity in the world. This is the case, for instance, when the empirical evaluation of international policies results in straightforward judgements of the efficacy of these policies, their ability to achieve their stated objectives or when economic modelling attempts to predict and prepare for different scenarios unfolding in financial markets.
Figure 2: Europa, image source: NASA https://www.jpl.nasa.gov/images/jupiter/europa/20141121/pia19048-16.jpg
However, many cases presented at the workshop demonstrated that ambiguity is not reduced but flourished as a result of trials and reflected on the resulting need for this to be negotiated or managed. This negotiation is sometimes fundamental and puts at stake not only the outcome or viability of the subject of testing, but also its very purpose, organizational context and knowledge-producing capabilities. Take the case of Janet Vertesi (Princeton) study of space exploration and NASA’s planned efforts to investigate Europa, one of the moons of Jupiter. Vertesi showed how the scope of the mission was being constantly reconsidered due to evolving interests of actors involved in it and ongoing organisational, political and budgetary changes. Moreover, Europa — a silent (so far), distant and, some scientists argued, “uncooperative”, object of study — itself possesses multiplicity insofar as different versions of this moon came to be defined during laboratory testing in preparation for the mission. In a very different case, it was suggested that this sort of “hyper-multiplicity” is sometimes deliberately built into tests, as in the case of underground electronic music, studied by David Stark and Giovanni Formilan (University of Warwick) where it provides means for the development of personas, creative identities and projects. In this context, ambiguity is considered an asset that allows for multiplicity and increased freedom of expression and thereby, perhaps paradoxically, enhances control over the things that are not supposed to be tried out within given a context. When electronic musicians assume aliases, they often try to escape judgement based on their previous output. Even more importantly, they want to direct attention to the music and thereby protect authenticity as an artistic value that they seek to convey. These examples suggest that ambiguity can be an outcome of a test as much as its precondition and as such can be manipulated, negotiated and intentionally minimized. The generation of multiplicity (of actors, objects, relations) is a constant feature that those implicated in testing must presuppose and deal with.
Another key, related, issue is the productivity of tests. The workshop reflected on this in several ways. For instance, there are internal, procedural questions about the possibility of conclusively defining the actual answers that tests give in situations of uncertainty. The productivity of a test depends partly on well-designed definitions of its scope, aim, actors and objects engaged and acceptable (or imposed) consequences (test as constraint), but, equally, on the unfolding test to problematize these (test as event). Regarding the latter, in the course of the test process itself, objects and actors may reveal their unruly nature, unbreachable barriers between human and non-human world may be breached, faults revealed in operational definitions, or the fact surfaced that the test brought hidden agendas into play. The latter dynamic can be seen in the “citizenship testing” in the Netherlands, discussed in the presentation by Willem Schinkel (Erasmus University Rotterdam). This ostensibly formalized procedure certified by the state and based on a legal framework is at the same time implicated in a highly contentious political situation and long-lasting public discussions about national belonging. Another example is street trials of autonomous cars, discussed by Marres (University of Warwick) where computational experimentation interacts with ordinary practices of road users in real-life urban spaces. Streets of Greenwich, Milton Keynes or Coventry become places where new technological prototypes and visions are introduced to society, and where, consequently, questions about rightful membership in society are being posed.
The case of social credit system in China, examined by Jonathan Bach (New School for Social Research) is very informative in this respect, also because it is embedded in the everyday life of a large population. As an interactive, evaluative arrangement, this system has not only the obvious result of identifiably locating individual persons in rankings that are visible to others, it is also a means to secure docile citizens, social order and a controlled population, by creating what Chris Anderson (Leeds University) called a “total test environment”: subjects do not know when and how they are evaluated, and so are continuously, and comprehensively, put to the test. The testing system is then also very clearly a place for the production of institutional state power. This goes a far way beyond the particular situation of testing and brings into view questions of authority, social reproduction and long-term systemic consequences of testing.
The sociological understanding of testing needs to be sensitive to these different “testing” consequences and empirical analysis must be able to detect their short and long-term effects. This task is complicated by challenges of access to the test situation, and because it requires taking into account wider circumstances in which a test is enmeshed, which may be difficult to observe, and even so more because testing is a device that operates by creating and combining different temporalities. But this can be partly surmounted by asking what the precise moments of deployment are, what is the expected and actual duration as well as whether tests are continuous and/or repeatable. Another issue has to do with the fact that tests can bring the future to the present — or, they are mechanisms of “futuring”. Their strength then derives from both their momentous decisiveness and their ability to bring the future closer to reality.
There are also very clear political aspects of testing that, although invoked during the workshop, need further exploration. Given the wide proliferation of tests in contemporary societies, it is worth asking not only whose interests are served by these processes but also what entities enter and are amendable to testing procedures and which are exempt or excluded from them. As a few of the presentations showed, it is not entirely clear what tests enable, above and beyond other forms of accreditation, in terms of creation of individuals, communities or technologies, not to mention knowledge. Yet they are today one of the main avenues through which societies characterise themselves and their members. As such, the under-defined nature of a test as a partial probe reveals some of major tensions and contradictions we face today as social scientists and citizens. The capacity to produce meaningful, engaging tests for technologies, policy propositions and social actors is likely to remain of importance to the operations of the modern state and society. Important questions raised during the workshop — about the possibility of self-reflection, uncertainty, of critique and emancipatory potential in and of testing — will have to be revisited.