In today’s audiovisual media culture, the combination of images and sounds is omnipresent and a matter of course. The technical and esthetic boundaries between the two have become fluid. Digitization has broken down auditory and visual information into bits and bytes and thus made it possible to link it or translate it at will. In a parallel development, a theoretical interest has emerged in concepts for combining images and sounds, uniting the arts and their history. In retrospect, the desire to fuse hearing and seeing appears in fact to have been a cultural constant for millennia, which can be traced back to ancient models for analogies between colors and sounds.
Since the end of the nineteenth century, with the rise of technical media, efforts have been made to reproduce, intensify, expand, and finally to transfer human audiovisual perception to virtual worlds that promise to fulfill utopian fantasies of synthesis. In this tension between naturally and artificially (or artistically) produced audiovisuality, between its immersive or analytical implementation, the question arises all the more: To what extent are we actually conscious, or can be made conscious, of what happens between the auditory and the visual?
The convergence of the audiovisual is not just a matter of technology; research into perception also increasingly calls into question the separation of the senses. At the same time, however, there is no genuinely audiovisual discourse about hybrid audiovisual artworks themselves: now, as ever, they are usually evaluated in a specific context, localized, for example, either in the visual arts, in music, in film, or in club culture. The rules of criticism not only conform to different standards in each case, but also often ignore one side of the audiovisual construct. Even if media culture has become multisensory, an established ‘in between’ that could do justice to hybrid art forms does not yet exist.
It is probably impossible to explore the diversity of this theme using only a single approach that brackets out other important aspects in the process. The overarching project SEE THIS SOUND thus presents different approaches as modules for this sort of multiperspectival representation: an extensive exhibition and an accompanying catalog1, a symposium 2, two scholarly book publications3, and a comprehensive web archive, which documents and networks all of the aforementioned areas.
In the humanities, too, responsibility for examining the relationships between images and sounds continues to be divided among different specialties operating in isolation, often finding room only on the margins. Specific forms of artistic expression or historical phases in the relations between sound and image in art, in the media, and in perception have been the subject of numerous in-depth scholarly studies.4 Similarly, extensive exhibitions have been dedicated to the history of the correspondences between music and the visual arts. 5 Even so, a fundamental overview of the multiple artistic, technical, and perceptual perspectives on the connections between auditory and visual phenomena has not been produced to date.
The present compendium of audiovisual culture should be seen as a contribution to filling this gap in that it bundles information from individual disciplines and offers a comprehensive foundation of knowledge on the relationships of images and sounds. The work includes thirty-five articles that present not only the spectrum of audiovisual art forms and methods for combining auditory and visual phenomena, but also the modalities of their perception. Thanks to this combination of longitudinal sections of cultural history and systematic cross sections, the field of the audiovisual arts and phenomena is opened up in two ways. Articles written by experts in art history, musicology, film theory, theater studies, media studies, software studies, the media arts, the psychology of perception, and neurology focus on their respective fields, but are constructed in such a way that they touch on, carry forward, and refer to one another.
Aspects that cross thematic boundaries and could only be outlined in the historical presentations are examined more closely in longer essays. For example, the publication includes essays dedicated to the relationship between pop culture and the visual arts, the role of technical equipment in the process of artistic production, and the creation of situations for specifically audiovisual experience and their bases in the psychology of perception.6
All of the texts are also supplemented with analyses of exemplary works, which illustrate general developments by discussing specific implementations.
Thanks to this concept, the online presentation offers an opportunity to approach the theme from multiple perspectives. Not only are the individual contributions interlinked (where their themes intersect) and associated with relevant analyses of works, but broader connections between them are also revealed.
The chronological structure of the historical articles on audiovisual art forms makes it possible to characterize, section by section, the relevant temporal horizon discussed in each case and reveals not only the diachronic perspective of each individual text but also a synchronic point of view that transcends the text.
Keywords specific to the themes establish conceptual connections both between the individual art forms and between the technologies and modalities of perception on which they are based and also raise overarching questions.
In addition, a database in which all of the relevant entities within the individual articles—e.g., people, works, institutions, and events, as well as software tools and hardware—are compiled and interlinked not only permits comprehensive, intertextual research, but also offers an overview of the contexts in which these appear.
This approach is intended, on the one hand, to do justice to the cross-genre networking of art scenes and discourses (which cannot be addressed adequately in the individual presentations) and to the inter- and multidisciplinary activities of individual artists, but also to reveal cross-connections that often remain undiscovered when the focus is restricted to a single discipline.
Only this type of bundling and networking of the perspectives of various specialties makes it possible to offer a comprehensive picture of the wide-ranging themes. Ultimately, however, even this approach can only offer a series of signposts in the terrain of ‘audiovisuology.’
An established discipline of ‘audiovisuology,’ as suggested by our title, does not (yet) exist. It takes shape here initially as an intersection—or, better, sum—of the thematic areas treated in the individual essays, while the intention of the overall project publication is to provide impetus for further development. Taken as a whole, the publication reveals audiovisual research at the point of intersection between art, science, and technology to be one of the great fields of experiment in the modern era—one which, like the project of modernism as a whole, has not been and cannot be brought to a close. The heart of this striving for interdisciplinarity and innovation that transcends genre calls into question the concepts of the work of art and the models for exploitation of various disciplines and thus itself repeatedly manages to avoid being categorized and canonized.
Dieter Daniels and Sandra Naumann