Inputs-outputs, sensing, experience and visceral loops

I attended the Inputs-Outputs conference at Brighton earlier this week (many thanks to Harry Witchell, Carina Westling and the team for organising such a stimulating event).

I very much enjoyed the variety of talks from different disciplinary perspectives. Several focused on how systems might ‘measure’ different facets of human interaction and communication, for example Alessandro Vinciarelli discussed detecting conflict from TV footage of political debates, Fabien Ramseyer spoke about detecting synchrony in videos of psychotherapy sessions, while Effie Lai-Chong Law addressed the broader question of what aspects of user experience might actually be measurable.

To complement these were talks that focused on the design of powerful or intense experiences, including Colin Nightingale from the theatre company Punchdrunk describing their distinctive brand of immersive theatrical performance. In this vein, my own talk presented our recent work on uncomfortable interactions, drawing on the performance works of Blast Theory and the thrill experiments of Brendan Walker to illustrate the deliberate use of various forms of discomfort to create entertaining, enlightening or socially bonding experiences.

This mix of perspectives got me reflecting on the relationship between the design of experiences and the measurement of people’s responses to them. There is a range of technologies available for measuring response, from wearable biosensors that detect heart rate and the sweatyness of one’s skin (that might potentially reveal anxiety of arousal) to the use of video cameras (including depth cameras such as the Kinect) to detect facial expressions, postures, synchronous movements and other bodily responses that people may consciously or unconsciously reveal. More ambitious still, is work on affective computing that tries to analyse such signals in order to interpret some sense of emotional response.

A common idea is create experiences that adapt to these signals, for example games that respond to levels of excitement or otherwise, or perhaps even therapeutic experiences that try to adjust mood.

However, I feel that there are other potentially important approaches that deserve consideration. I’m especially intrigued by the possibility of flipping things around, requiring people to adapt to the systems rather than having the systems adapt to them. This would mean humans generating and maintaining expressions and postures, or even heart rates and sweatyness, so as convince a system that they are in some kind of desired physiological or possibly emotional state. Could a player convince the game that they are suitably calm or perhaps angry so as to unlock a particular power-up or storyline or action?

Interestingly, the measurement system does not have to be perfect here – it may not matter that the player is actually ‘calm’ – but rather that it is sufficiently intelligible that they can play with it and ‘fool’ it into think that they are . A key challenge for the player may then be in maintaining this illusion over time, not letting the mask slip even when provoked by the experience. In effect, the measurement system  introduces constraints around which the player must work – and of which they become aware  – rather than trying to seamlessly measure their response. A powerful side-affect, or perhaps even direct goal, may  be to make people more aware of their own physiological responses, focusing them inwards on their own feelings and bodily experience.

An interesting example of this is the Broncomatic, a breath-controlled bucking bronco ride whose movements become more the extreme the more its riders breathe. This sets up a powerful tension in which riders have to battle for control of both the ride and their own bodily response (which in turn is influenced by the ride), with some becoming very inwardly focused and self-aware of their own breathing.

It would be interesting to explore other kinds of visceral feedback loop in which humans are coupled to systems that both sense and actuate them, where both parties battle for control, and where participants potentially become more self-aware of their own emotional and physiological behaviours.


New research blog underway

Welcome to my new research blog. I’m aiming to gradually build this site over the next few months to become a resource for accessing papers, videos, slides, details of projects, news, discussions and other materials related to research into Human-Computer-Interaction, interactive media, and creative and cultural applications of computers. It will take a while to build it up – in fact it will probably always be a work in progress – but I hope you find something interesting here.