The enactive paradigm for interactive narration

Today’s technologies, increasingly invisible and pervasive, are able to know us better than we can imagine. The enactive paradigm in such a context can be applied to produce data in real-time. The main question such research asks is whether this data can be useful in an interactive narrative context for the design of enactive storytelling experiences. The research explores how the enactive paradigm can be applied, using eye-tracking and emotion-analysis technologies as input for enabling enactivity.

The thesis focuses on interactive streaming experiences as a promising field of investigation for this new paradigm of interaction. Thus, seeing interactive storytelling as a tool for the investigation. The present research supports the idea that interaction design, with its perspective and practices, can be a way to explore these issues, and contribute to the development of new products and services as well as to the exploration of new paradigms, and therefore can provide a contribution for the development of society.

Year

2020

Reference Lab

IEX Interaction & Experience Design Lab

Project Team

Supervisor: Professor Laura Varisco

Student: Giulio Interlandi

Aiming at examining technologies to be involved in the creation of enactivity, the first prototype represented a starting point for the exploration and identification of potentialities of the use of eye-gaze as an input for controlling an interactive movie experience.

Our challenge was to “design an interactive experience to face how new technologies can know you better than you. Much better.” To accomplish this challenge we have therefore chosen to use an eye-tracking device to let the user interact and affect the storyline unconsciously. The what of our project was a catchy kiosk for which we created some graphics and communication strategies, such as positioning it next to each big tech company office based in Milan (Apple, Google, etc.). Then the why was to raise awareness about how personal information is collected today. In our how, people will go through a crescendo of restless emotions, only in the end, we will reveal them the back scene of the experience, how we were able to collect their personal information by following their eyes. We also defined a target, designing and delivering this project to people with digital attitude (who). In our concept, users were invited to enter the kiosk, unconsciously interact with the hacked version of Bandersnatch, and discover the final message and the rationale behind the experience. The main purpose of the first exercise during the workshop was to speculate and raise awareness in the spectators about both the possibilities and the issues, in terms of privacy, of modern technologies’ trend of increasing automation in providing content.

The iterations of interactive prototypes and the investigations produced initial guidelines aimed at creating design areas of experimentation identifying the starting point for possible future directions. Moreover, the knowledge produced in this work can be considered as a quantum leap in design awareness, including the enactive paradigm in a context of design process and research through design.

There are three possible guidelines identified by this study: Direction of change, Areas of focus, and Unexpected interactive elements. These possible guidelines can be intended as an initial cue, they are not definitive guidelines but they can be an excellent starting point for an approach to enactivity.

Direction of change. Aligned with Tikka’s (Tikka et al., 2006) principles, consider the increase or decrease in the values to affect the emotional mood through the management of the narrative flow including or excluding scenes and other ontologically organized content. Positivity or negativity of the valence and the value related to arousal can be particularly useful to detect the direction of change, especially for scenes with a predominance of emotional characteristics.

Areas of focus. The eye-tracking can provide a further focus on the user’s gaze when the emotion analysis is performed. In scenes with a high visual component, the eye-tracking data can be crossed with the facial ones to understand the user’s emotions with respect to a single eye-area. This means that from the author’s or designer’s point of view it is possible to know the valence and arousal, as well as the single emotion involved respectively to a visual element of the montage. Therefore, there is the possibility to compare different visual elements both in terms of duration of the fixation and in terms of comparison of emotional analysis. Hence psychology and bio-marketing studies can be useful in the design of emotional triggers and could provide an analysis of the like or dislike of the visual elements.

Unexpected interactive elements. In order to open new scenarios, it is possible to work on placing visual “Easter-eggs” within the enactive experience. Thanks to eye-tracking technology, it is possible to know if a single visual element has been looked at by the user and for how long.

These guidelines are to be understood in a broad sense, as possible directions to use from the designer’s point of view to produce enactivity. There is a strong need to support the project with knowledge about biosignals, psychology, and neurology. Also from an artistic point of view, these guidelines can be useful in the creative field in areas even very different from the cinematographic one.