Dr. Will Pearson

School

University College Dublin

Keywords

inclusive design, sound arts, sonic arts, social semiotics, cognitive psychology, computer science

Abstract

This thesis explores the relevance that experimentation and research in sound/sonic arts1 have for redefining listening experiences on mobile devices. To this end, the thesis takes the conceit of developing a theory of auditory saccadism (here referring to the unconscious involuntary movements made by the eye back and forth across text in visual reading). It proposes to benefit designers of interactive media who use sound as a significant element of their work.

From tentatively naming and identifying the possibility of an auditory imagination, the thesis centrally considers if there is an optimum format for delivering auditory-led reading experiences on mobile devices. The thesis asks: what new areas of content and narrative best suit those designers whose creativity and own uses of mobile technology led them to consider designing for this platform? What combinations of image, sound and interface work best to engage the imagination and replicate the intentions of the author in the minds of an audience? Developing these research questions further, could there be an equivalent in audio terms for the saccadic nature of the visual reading experience? The thesis builds upon reference to the practical lessons to be learned from artists who have used sound in their work, and refocuses on the ways in which sound can be used more innovatively in the future, and on the ways in which rhythm, silence, repetition and other qualities can be used alongside visual storytelling techniques. It is hoped that this work and its original contributions, including the original interviews and other data and digital assets produced, will be of interest and value to mobile developers and sound artists, as well as to the wider Inclusive Design and Creative Technology Innovation communities in both practice and academic scholarship.