1 Predecessors and Pioneers of Audiovisual Software Art

There are thousands or perhaps even tens of thousands of audiovisual software arts practitioners today, and yet the origin of these practices sprang from the work of just a handful of artists who obtained access to computer laboratories in the mid-1960s. The work of California-based filmmaker and animation pioneer John Whitney is a reasonable starting point for this history. Whereas most of Whitney’s contemporaries in early computer art (such as Georg Nees, Frieder Nake, Manfred Mohr, and Chuck Csuri) were focused on software-generated plotter prints, Whitney was interested strictly in the music-like qualities of time-based imagery. Computers in the 1960s were too slow to generate complex images in real time, however, so Whitney instead used the computer to output frames of animation to film. In animations like Permutations (1966–1968; developed in collaboration with IBM researcher Jack Citron) and Arabesque (1975; created with the assistance of Larry Cuba), Whitney explored the ways in which the kinetic rhythms of moving dots could produce perceptual effects that were strongly analogous to modulations of musical tension. Whitney’s films from this period were generally accompanied by nonelectronic music;^[1] only later, with the advent of personal computing and real-time graphics in the early 1980s, did Whitney’s focus shift to the development of a software instrument with which he could compose imagery and music simultaneously, as demonstrated in his animations Spirals (1987) and MoonDrum (1989).

Although the late 1960s and early 1970s witnessed rapidly accelerating technical advances in computer graphics and computer music, the capacity to generate both media in real time was still several years away. For this reason, many significant experiments that would lay the conceptual groundwork for purely computer-based real-time audiovisuals were nonetheless carried out in the off-line context of the film studio. One example is the work produced at Bell Laboratories by the American computer artist Lillian Schwartz (b. 1928), who collaborated with notable computer musicians on abstract film animations like MATHOMS (1970; with music by F. Richard Moore), MUTATIONS (1972; with music by Jean-Claude Risset), and MIS-TAKES (1972; with music by Max V. Mathews). Later, some artists combined computer-based synthesis in one medium with traditional live performance in another. German graphics pioneer Herbert W. Franke, writing in 1975, described the production of two ten-minute graphic music films (Rotations and Projections, 1974), in which live jazz musicians freely improvised in response to simultaneously projected patterns of abstract animated lines.^[2] In Schwartz’s 1976 performance, ON-LINE, a live dancer and musicians performed in real time while Schwartz, playing a QWERTY keyboard, created special graphic effects on a computer-controlled video system.^[3]

Possibly the first computer system capable of synthesizing both animation and sound in real time was the VAMPIRE (Video and Music Program for Interactive Real-time Exploration/Experimentation) system developed by Laurie Spiegel between 1974 and 1976 on a DDP-224 computer at Bell Laboratories in New Jersey. VAMPIRE offered a drawing tablet, foot pedal, and a large number of continuous knobs and pushbuttons—all of which could be used to modulate and perform a wide variety of image and/or sound parameters.^[4] Built on top of Max Mathews’s computer music research system, GROOVE (Generating Real-time Operations on Voltage-controlled Equipment), VAMPIRE, according to Spiegel,

was an instrument for composing abstract patterns of change over time by recording human input into a computer via an array of devices the interpretation and use of each of which could be programmed and the data from which could be stored, replayed, reinterpreted and reused. The set of time functions created could be further altered by any transformation one wished to program and then used to control any parameter of image or of sound (when transferred back to GROOVE’s audio-interfaced computer by computer tape or disk). Unfortunately, due to the requirement of separate computers in separate rooms at the Labs, it was not physically possible to use a single set of recorded (and/or computed) time functions to control both image and sound simultaneously, though in principle this would have been possible. ^[5]

Other significant software-based or software-generated audiovisual art of the late 1960s through the early 1980s are, among others, the computer animations of Stan VanDerBeek, Ken Knowlton, Tom DeFanti, and Larry Cuba; the computer-controlled laser projections of Paul Earls and Ron Pellegrino; and the interactive installations of Myron Krueger and Ed Tannenbaum. The introduction of the personal computer significantly broadened the landscape of audiovisual arts, making way for new forms like the digital video performance work of Don Ritter and Tom DeWitt, and the interactive desktop software creations of Adriano Abbado and Fred Collopy.