A day of looking at interdisciplinarity through an interdisciplinary lens

Maria Petrescu (Centre for Interdisciplinary Methodologies)

On February 6th, the Indicating Interdisciplinarity in AI workshop brought together partners of our collaborative project Inventing Indicators of Interdisciplinarity, developed by Noortje Marres in collaboration with colleagues at the Centre for Science and Technology Studies (CWTS), University of Leiden. Aligned with one of our core research themes, Interdisciplinary methods and methodologies, the project aims to find ways for indicators to be more than “instruments of measurement”, but sites of methodological innovation, participation and creativity which would “not bracket but enable the exploration of the differences and asymmetries between disciplines” (Lury, Lammes, Uprichard et al, 2018).

In the first talk after the introduction, Tommaso Venturini (Centre for Internet and Society, CNRS) presented his work (in collaboration with Axel Meunier from SciencesPo médialab) for the Global Forum on Artificial Intelligence for Humanity (Paris, 2019). In order to frame Humanity as more than “general ideas of ethics”, but as including humanities and social sciences, they produced a “mapping of the field of AI”, encouraging dialogues between natural sciences, social sciences and information sciences. By visualising networks of co-citation (pairs of academic articles cited together in a third paper) for articles and conference proceedings from ISI Web of Science and superimposing metadata (including authors and institutions related to each article), they presented the public with possibilities to recognise themselves, their academic departments, and emergent disciplinary areas, and enabling new ways of interpreting the maps. Specifically designed to be a “political intervention in a political space” (as Venturini later clarified in the brief Q&A session), this work brought several design “twists” which I believe deserve special mention, starting from forgoing the suggestion to project the maps on an interactive screen and using not-so-high-quality paper and highlighters instead, thus abandoning the god-like viewpoint implied by traditional network visualisations with clear node/edge delimitations.

Figure 1 Tommaso Venturini (CNRS)

Following this, Ludo Waltman (CWTS) initiated a provocation for us to reflect together on how questions about interdisciplinarity and AI can be answered from a scientometric point of view and how we can use bibliometrics more productively. With a clear emphasis on the black-boxed nature of the act of field delineation (he later noted his own refusal to delineate the field of AI), Waltman suggested a move from the traditional bibliometric approach (as shown in the pipeline in Fig. 2) to a model where the last two steps are replaced by indicators of scientific fields and mapping of scientific fields (relationships between fields), respectively. In the end, Waltman clarified that even if the acts of labelling undertaken along the pipeline are subjective, his aim is to “provide tools” to facilitate discussions about what AI might mean and thus to “arrive at a number of different perspectives about what AI could be” (emphasis added).

Figure 2 Ludo Waltman (CWTS)

Just as thought-provoking as Venturini and Waltman’s presentations, the brief discussion that followed raised a broad range of pertinent considerations. These included comments about the potential usefulness of dynamically visualising the evolution of field delineations over time (Alexander Stingl, CIM) and notes on the possibility to delineate “the AI that we care about now” by “following the historical track of connectionist resurgence” (Michael Castelle, CIM).

Throughout this discussion, however, the constructedness of disciplines, interdisciplinarity and of measures of interdisciplinarity themselves emerged as the leitmotif. This became apparent in comments on the researchers’ role in labelling disciplines (Cagatay Turkay, CIM), on interdisciplinarity potentially becoming invisible if the research process begins from already-delineated fields (Matt Spencer, CIM), in clarifications of the value of interdisciplinarity as highlighting “knowledge in formation” (Prof. Noortje Marres, CIM), and in Tommaso Venturini’s remark (which elicited a round of smiles in a room full of researchers of interdisciplinarity) that “there is no such thing as interdisciplinarity” because “disciplines are constructs” and we are all “playing a language game of creating boundaries” (emphasis added).

After this discussion, Ismael Rafols (CWTS) moved the discussion further, into the realms of decision-making. More specifically, addressing the question of “How can we help decisions be made while preserving diversity?”, he explained how the trade-off between conventional indicators and mapping (of interdisciplinarity, in this case) is one between pluralising the discourse on the one hand and helping close down the debate, on the other hand. He later clarified that he is not interested in fostering closure, but in helping each user find their own closure. As such, he proposes using maps “together with the decision makers” and approaching them from “the toad’s view”, thus “produc[ing] an understanding which is specific, but which can be traced to a series of subjective choices” (emphasis added). In this process, one could start by learning about the different understandings of interdisciplinarity for such maps. Is it about the distribution of different bodies of research across diverse topics? Or about the engagement between distant methodologies? Or between disciplines which are very far apart? Through this type of questions, indicat-ing (as a verb) becomes a way of helping users and decision-makers dive into the map.

Figure 3 Ismael Rafols – from mapping to indicating

The lunch break was followed by a round table discussion with presentations by Rodrigo Costas Comesaña (CWTS), CIM’s Greg McInerny and CIM academic alumna (now at Leiden University) Prof. Sybille Lammes, moderated by Thomas Franssen (CWTS).

Rodrigo Costas Comesaña started the session by introducing altmetrics and advocating for a move beyond science-science interactions, towards looking at science-society interactions. In this regard, altmetrics represent a potential enrichment of typical metrics with data gathered mostly from social media (Twitter, LinkedIN, blogs et cetera). In this scenario, the type of insights derived from Ludo Waltman and Tommaso Venturini’s projects would be taken beyond the realm of scientific literature by detecting “heterogenous couplings” in social media debates about science and innovation, articles shared and tagged (for example, in the same tweet or by the same account) and so on.

Greg McInerny continued the discussion by bringing to the foreground assumptions made when visualising bibliometric data, especially in the form of networks. More specifically, he pointed out that in this type of visualisation it is impossible to see what is missing in the data (i.e. network visualisations are built around the existing data, and space/clustering are but functions of data and algorithms). Then, drawing on Barry, Born and Weszkalnys’ (2008) Logics of interdisciplinarity, he suggested looking closer at the different types of interactions (synthesis, subordination and agonistic-antagonistic) between disciplines, as opposed to entire networks of knowledge. In the end, he proposed that Visualisation should not be subordinated in this type of work and that conceptualising the types of interdisciplinary interactions would be a great source of enrichment.

Figure 4 Prof. Sybille Lammes (University of Leiden)

For the last presentation of the round table, Prof. Sybille Lammes considered bringing play back into methods. With an analogy drawn from a conversation on deep neural networks, in which the layer right before the end result was pointed out as the most interesting level of the process (“where there’s still a bit of mess[…] at the level where there’s still play”), she argued that we should make room for ludo-epistemology to bring out different ways of interdisciplinarity which are “less stable and less fixed”.

Perhaps the highlight of the workshop, the prototyping exercise led by Sarah de Rijcke and Tjitkse Holtrop (CWTS), presented us with three case studies of AI deployments (self-driving cars, AI for mammography data, and face recognition). In small groups, we were encouraged to look at the unfolding of these cases in the news and to discuss specific events by adopting fictional roles including those of observers, engineers, regulators, industry and user representatives, as well as public philosophers. This shifted our attention to the disciplines being brought into (and also ignored by) public discussions and discourses, how attitudes and alliances between specific types of actors emerged naturally in each group (for example, between industry representatives and engineers) and how interdisciplinarity is present (and indicated, or not) in each case.

In the final discussion, CIM Prof. Celia Lury insisted on the importance of considering the questions of What do we want from indicators? and What do indicators want from us? together, pointing out that throughout the workshop discussions two understandings of objectivity were deployed (as “objective representation independent of the observer” and as “shared reference point”). Stemming from this, the room launched into a conversation about the generalisability of indicators, the need (or lack thereof) for indicators to be stable, the language of interdisciplinarity (and on how good indicators of interdisciplinarity should achieve a certain polyvalence).

Personally, I think some of the open-ended questions that emerged in the last few minutes of this conversation are the best take-away and the natural choice for summarising the event:

  • If we don’t want indicators to feed into rankings, what do we want them to feed into? Trajectory of choice? Visualisations? Games? Stories?
  • How, if at all, would these help interdisciplinarity evolve?
  • If we care about interdisciplinarity, how can we sustain it? (and here I will include a somewhat amusing answer someone whispered loud enough for only a few of us to hear: “Perhaps by not indicating it at all?”)