Luis Rocha Antunes - The Multisensory Film Experience: A Cognitive Model of Experiental Film Aesthetics

First published in the UK in 2016 by

Intellect, The Mill, Parnall Road, Fishponds, Bristol, BS16 3JG, UK

First published in the USA in 2016 by

Intellect, The University of Chicago Press, 1427 E. 60th Street,

Chicago, IL 60637, USA

Copyright 2016 Intellect Ltd

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission.

A catalogue record for this book is available from the

British Library.

Copy-editor: MPS Technologies

Cover designer: Jane Seymour

Production manager: Jelena Stanovnik

Typesetting: Contentra Technologies

Print ISBN: 978-1-78320-628-5

ePDF ISBN: 978-1-78320-629-2

ePUB ISBN: 978-1-78320-630-8

Printed and bound by Short Run Press, UK

The book is published with support from The Faculty of Humanities,

Norwegian University of Science and Technology.

To my parents, Antnio and Isabel, for their unconditional love and support.

To Cynthia Freeland, philosopher, lover of cats, film and the arts. Woman of courage.

To the Portuguese Foundation for Science and Technology, whose doctoral funding kept this project alive.

We think we know by the look of a thing what we originally learned by feeling it.

(Yeo, 1884: 536)

I have a vivid memory from my early childhood. My grandfather drove my brother and I to the Canadian National Exhibition in Toronto, where I saw my first IMAX film. The large format screen, several stories tall, filled my entire field of vision, and speakers played sounds so loudly they shook my small body. The movie, showing the point of view of aircraft flying over various terrains, was little more than a demonstration of sensations this new technology could deliver, but to me it was nothing short of magic. Every time the plane banked to execute a turn, I leaned in my seat to keep my balance. As the plane glided past the edge of a cliff, I felt my stomach drop.

I remember this as the moment when my love of film as a sensory experience took root. I added to my early viewing experiences movies like Star Wars, Close Encounters of the Third Kind and The Empire Strikes Back. Yes, I empathized with characters, thought through plots, and connected themes to memories of my own experiences, but I also felt these films at the perceptual level. They engaged my reflex action when a villain jumped into view, made me queasy when a TIE fighter spacecraft spun out of control and caused me to shiver when the hero was caught in a blizzard on an icy planet.

We do not think about these experiences. We do not cognate that we should feel pain. Instead, arrangements of a films visual and audio components engage our perceptual systems. Practiced artists and craftspeople working in film production know these tricks. We tilt the camera slightly to create a feeling of unease, use colour palates to warm or cool the audience and show characters reaction to pain to stimulate that same feeling in those watching the screen. Years later, as I taught the aesthetics of film to my production students, I was at a loss to explain how these techniques worked. If we only see and hear a film, how are these other experiences generated? Using the phrase the magic of cinema seemed to me lacking.

Luis Rocha Antunes provides an answer in this book. Using new understandings in the neuroscience of perception, he argues that audiences experience a film with all their senses and create perceptions before the conscious mind has an opportunity to connect metaphors or make sense of the experience. Neurologists have long recognized that humans possess more than the commonly recognized Aristotelian five senses of sight, sound, smell, taste and touch; nociception, proprioception, thermoception and the vestibular sense all provide the brain with distinct feedback. These senses interact with one another, contributing to a multisensory perception of an environment. In fact, we do not see and hear a film, but perceive it with all our senses working in an ecological relationship.

The Multisensory Film Experience joins a growing tradition of theory within a field known as cognitive film studies. Beginning with David Bordwell, Nol Carroll and Barbara and Joseph Anderson, this line of scholarship reaches back to Rudolf Arnheim and Hugo Mnsterberg writing at the birth of cinema. Today, film scholars such as Murray Smith, Carl Plantinga and Torben Grodal have used neuroscience to reimagine theories of cinema, and researchers like Jeffery Zacks, James Cutting, Tim Smith and many others are examining how cinema taps into a human perceptual system that evolved within a natural environment. Antunes continues this shift of understanding cinema from a system of arbitrary signs to one rooted in evolutionary perception.

Applying his theory of a multisensory film experience to the work of three film auteurs, Antunes demonstrates how stimuli of light and sound engage and reverberate within a multimodal and multisensory perceptual system, generating perceptual experiences beyond sight and hearing. I look forward to experiencing these films again with the knowledge that Antunes has provided. I believe you will as well after reading this book.

Michael Grabowski

I owe to my parents, Antnio and Isabel, the existence of this book. Their unconditional love and support gave me the fundamental motivation and courage to make this book come true. I am grateful and forever indebted to a number of individuals and organizations who, directly or indirectly, supported and invested in this project, inspired me, contributed with their work or feedback and kept me going with their friendship or with their professionalism, namely, and above all, to my advisor during these five years of research leading to the materialization of this book, Virginia Pitts, but also to Steve Harries, Jelena Stanovnik, Shankha and Varsha Sarkar, Johan Magnus Elvemo, Lasse Hodne, Harriet and Jim Costa, Paul and Chris Greenfield, Ali Shirzadi, Margrethe Bruun Vaage, Eirik Frisvold Hanssen, Gunnar Iversen, Anne Marit Myrstad, Bjrn Srenssen, Cynthia Freeland, Joseph Kickasola, Michael Grabowski, Monica Mecsei, Hanne Siri Sund, Christer Bakke Andresen, Luann Jennings, Slvia Pinto Bachmann, Mariam Goshadze, Heather Williams, Ragnhild Fjellro, Jelena Rosic, Nastupaet Vesna, Stephan Klingebiel, Tamer Te, Eliane de Larminat, Laura Marks, Trond Einar Garmo, Vemund Warud, Gro Lurs, Birgit Stenseth, Torben Grodal, Gunn Bekken, Jon Huseby, Barbro Rnning, Asbjrn Tiller, Stine nes, Toril Wahl, Ellen Karoline Gjervan, Guri Ellen Hanem, Inger Malene Nausthaug, Tanya Ronnen, Jonathan Hampton, Vittorio Gallese, Baltasar Bivar Branco, Carla Lopes and Filipe Morais, Carl Plantinga, Angela Whiffen, Ros Beeching, Frei Jos Carlos Matias, Marit Album Kvernmo, Joschka Schneider, mer Faruk rsn, Sofia Tonicher, Nuno Miguel, Linna Martn, Joanna Hearne, Cludia Castro, Rachel Cohen, Dariya Orlova, Wanda Bivar Branco, Margarita Bivar Branco, Helena Passanha, Maria Cndida Cerqueira Duarte Rocha, Jorge Rocha Antunes, Julian Garritzmann, Phil Tejeda, Sahar Zabihian and Behrad Rahmani, Mna Dnl, Jakub Sebastian Konefa, Ins Nunes, Seema Saujani, Moritz Riemann, Manila Castoro; the Portuguese Foundation for Science and Technology, the Norwegian Research Council, the Institute for Comparative Research in Human Culture in Oslo, the University of Kent, the Norwegian University of Science and Technology, the Faculty of Humanities (NTNU), the National Library in Lisbon, the Copenhagen University Library, the Portuguese Cinematheque in Lisbon, the Danish Film Institute Cinematheque in Copenhagen, the British Film Institute, Baylor University, Department of Art and Media Studies (NTNU), American Journal Experts, Intellect Books, Harvard Film Archive, the Art Library at the Calouste Gulbenkian Foundation, the University of Copenhagen, Harvard University, Department of Media, Cognition and Communication (University of Copenhagen), the Society for Cognitive Study of the Moving Image (SCSMI), the Society for Cinema and Media Studies (SCMS), the Society for the Advancement of Scandinavian Study (SASS), Department of Visual and Environmental Studies (Harvard University), the School of Arts (University of Kent), The Graduate School (University of Kent), the Norwegian Film Institute, the National Library in Oslo, and to others, anonymous or unmentioned, to whom I am eternally grateful and indebted.