Messy Methods and Moderation in User Tests

In this paper, we argue that practices of moderating usability and user trials are never as neutral, formal or procedural as most of the methods literature suggests. Borrowing from science and technology studies scholar John Law's notion of 'messy methods', we propose that moderating usability trials is always a 'messy affair' that involves unruly assemblages and requires the more or less ad hoc and in situ social and material management of rowdy relationships between different entities, such as human bodies and a vast amount of material objects. Based on in-depth and ethnographically oriented interviews with usability trial moderators, we analyse the skilled social, creative and embodied practices of moderators managing the realities of user test situations that are more complex than the literature renders visible. Our study shows that moderators employ a rich repertoire of techniques to master the situation. This points to the need for a revitalisation of the user test method to acknowledge the relationship between test person and moderator to obtain richer material. The paper contributes methodologically to human-computer interaction (HCI) and user test studies by developing the notion of messy moderation and messy methods in HCI. This holds significant implications for our view of user study methods applied and developed in HCI, as it invites sensitivity and reflexivity into how our methods contribute to creating the realities they aim to study.