Rhythm for the Ears and for the Eyes
Notes on Music and Film (4)
In a classical animated cartoon such as Tom and Jerry, action and music are closely coordinated. For this purpose, the images have been timed in advance to fit a steady beat – before there was any music. Watching the movie without sound, you may still be able to find the beat in the characters’ actions.
When we speak of rhythm, it is music we tend to think of first. ‘Visual rhythm’ almost sounds metaphorical, like ‘food for the soul’ or ‘mind gymnastics’. Applied outside music, the concept easily gets watered down to periodicity (as in the ‘rhythm’ of the tides or the seasons), or just any kind of patterning, temporal or otherwise.
And yet, why should rhythm primarily be auditory and musical, rather than visual?
The answer may not be immediately obvious. If rhythm is a regular flux of longer and shorter durations, of stronger, marked moments (beats) and weaker, passing moments (what lies between), why should we not find rhythm in what happens before our eyes?
In fact, we do; but mostly in those cases where the visible is closely coordinated with music: in dance. We dance to music; we hardly ever make music to a dance. The scarcity of dance without music is remarkable. It can be exciting, though, when the roles are reversed: when dancers manage to create at least the illusion of leading the music, as orchestral music may seem to emanate from the conductor’s dancing hands and arms. (Actually, it sometimes happens in the early cartoon.)
Movie Rhythm
In the 1920’s, when the synchronization of image and sound was technically problematic and aesthetically dubious, the model of music had a particular attraction to avant-garde film makers. This can be explained in part by a then fashionable fascination with synaesthesia, a search for modes of perception that are common to the senses (such as ‘hearing colours’). It coincided with a tendency towards abstraction in the visual arts. The abstract nature of music, particularly instrumental music, made it interesting as a structural model for an art of moving images.
The term ‘absolute film’ was sometimes used in imitation of ‘absolute music’, an odd expression that has its origin in nineteenth-century aesthetics. ‘Absolute’ means above all ‘abstract’: absolute music is supposed not to signify or represent anything else than what it is – a sequence of rhythmical tone patterns. That abstract-musical orientation is evident in 1920’s titles such as Film ist Rhythmus (Hans Richter), Diagonal-Symphonie (Viking Eggeling), and Lichtspiele Opus I-IV (Walter Ruttmann).
What gives abstract moving images structure is, as Richter’s title suggests, rhythm. It is striking that when these films are played in silence, music seems to be sorely missing; the images seem to lack both substance and motivation. But when music is added, it may be hard not to see the film as a kind of ’visualization’ of music, rather than what it should be: film with ‘musicalization’. A contemporary critic accordingly described a screening of Film ist Rhythmus as …
… a group of artistically reflected rectangles and circles, which performed drills and rhythmic gymnastics to music for a while. If the pianist became animated and started playing forte, the screen turned colorful, nervous, and jittery; if he tentatively stopped, the little squares slumped down, turning pale.
(Rudolf Arnheim about the historically famous ‘absolute film’ matinee in Berlin, 1925; quoted in Kaes et al. 2016: 460. The pianist must have had excellent timing!)
Sergei Eisenstein’s influential, though not particularly clear or consistent theory of ‘montage’ (editing) is also shaped by musical analogies: he distinguishes ‘metric montage’, ‘rhythmic montage’, ‘tonal’ and even ‘overtonal montage’. Metric montage is creating a succession of shots of particular durations, like bars of music (a procedure Eisenstein denounces as ‘mechanical’). On the other hand, …
In rhythmic montage it is movement within the frame that impels the montage movement from frame to frame. Such movements within the frame may be of objects in motion, or of the spectator’s eye directed along the lines of some immobile object. (Eisenstein 1977: 75, written in 1930; my emphasis.)
Modern film theory calls this ‘external’ (‘metric’) and ‘internal’ rhythm (Beaver 2006).
Another great Russian director, Andrei Tarkovsky, has criticized Eisenstein’s concept of montage, but understands this as ‘metric’ or ‘external’ montage; Tarkovsky’s own concept of cinematic rhythm agrees with Eisenstein’s rhythmic montage. The dynamics of this internal rhythm should be decisive in the editing process:
The dominant, all-powerful factor of the film image is rhythm, expressing the course of time within the frame.
The distinctive time running through the shots makes the rhythm of the picture; and rhythm is determined not by the length of the edited pieces, but by the pressure of the time that runs through them. (Tarkovsky 1994: 113, 120; my emphasis.)
‘Rhythm’ in this sense has more to do with speed or pace, and speed fluctuations – though the two are intimately intertwined, as well as with metre, the regular recurrence of ‘beats’. The distinction is clearly made in a more conventional handbook by the Hollywood editor and screenwriter Tamar Lane (with a sceptical jibe at theories of movie rhythm):
Tempo is the relative speed or rate of movement of the story as it unfolds from beginning to end. In some respects it may also be defined as the characteristic pace in which the situations or plot are told.
Rhythm is a regular or harmonious beat or accent to movement or sound; movement or sound marked by some regularly recurring accent or quantity.
Tempo – or pace, as it is sometimes called in the studios – is a well-known element in the cinema and was much utilized by some film craftsmen even in the days of the silent picture.
Rhythm, as it applies to the screen story, is an element that we sometimes read about, or hear discussed, in a profound manner. As far as the present-day cinema is concerned, however, it is a much overtalked and little practiced theory. (Lane 1936: 65)
Beeps and Bounces
Whether rhythm is more for the ears than for the eyes (and if so, why) is for music psychology to answer.
The earliest psychological experiments strengthened this idea of stronger auditory rhythm. When we compare a visual to an auditory metronome – flashing lights to clicks or beeps – beat perception is more confident with the beeps. Also, in hearing we perceive periodic, identical stimuli as not identical, but stronger and weaker. No clock ticks to our ears tick-tick; spontaneously and unavoidably, we hear tick-tock, strong-weak, a minimum of rhythmical (or metrical) organization. Whether the same effect occurs when we see, but do not hear the pendulum is a question the psychologists seem as yet to have ignored.
The visual disadvantage is less pronounced when beeps and flashes are replaced with simple animations such as a bouncing ball or, still more effectively, human body movement (Luck 2009, Iversen 2015, Ammirante 2016, Silva 2016). Much more rhythmical to our perception than a series of discreet on-off signals is a continuous, accelerating and decelerating process such as a natural bounce, or the simulated bounce of a conductor ‘beating time’.
It may be a different matter when we consider rhythm in a richer sense: not merely a pulse, but that endless variety of rhythmical qualities that we ‘feel’ rather than hear – the specific musical motion, gait, swing, pounding, flowing, stumbling, jumping, and so on; that seductive power of music that makes us move along, physically or in the imagination (and mostly subconsciously). It may be significant that the organ with which we sense our own bodily motion, balance and direction, the vestibular system, is located in the inner ear (Todd 2015).
But why, if musical rhythm so strongly evokes and provokes bodily movement, should it do so more effectively than movement itself? – I still think waltz music is a stronger stimulus to start waltzing than watching a waltzing couple.
Visualizing Music
Musical rhythm is not something we simply hear; it is perceived by deeper operations of cognition. It is not something a physical apparatus can register, say, by measuring loudness. Extracting even the downbeats from music by any system less intelligent than ourselves is a notorious conundrum in the fields of Music Information Retrieval (Durand 2015, Jia 2019) and music visualization.
This becomes evident when we observe the visualizer that will be familiar to most people, the one that comes with Windows Media Player. Watch this video: I bet you can’t ‘see’ the metre (time signature) of the music that has originated this piece of ‘absolute film’. How many beats are there in the bar?
(You will find the original sound file among the references below.)
Such visualizations seem in fact to have inherited the early twentieth century avant-garde’s synaesthetic aesthetic, in a crude form. As an automated kaleidoscope it is a perfect illustration to the founding document of ‘the doctrine of absolute music’, Eduard Hanslick’s famous-notorious Vom Musikalisch-Schönen (1854). The moving sound patterns of music, Hanslick argues, are like a child’s kaleidoscope, though “at an incommensurably higher level of realization”:
[Music] presents beautiful shapes and colours in ever-evolving variety, moving through gentle transitions and sharp contrasts, always symmetrical and complete in itself. (Hanslick 1854: 33, my transl.)
Well aware that his comparison of the masterworks of music with the chance patterns of a kaleidoscope is open to the charge of trivialization, Hanslick stresses its usefulness as a mere analogy: we should not attempt to give it visible form! Also, the Tonkaleidoscop “presents itself as an immediate emanation of an artistically creative mind”, not as a mechanical toy. But what precisely does it mean: “an immediate emanation of an artistically creative mind”?
Hanslick’s warning notwithstanding, it may be worthwhile to watch WMP for one or two minutes – if only to become aware of what it cannot do, how many musical dimensions elude this brightly coloured algorithmic randomizer. But above all: why does the motion of its abstract shapes not more closely follow the musical rhythm? I doubt that the designers were not interested. Though ‘visualizing music’ may be an absurd undertaking anyhow, if your aim is creating visual patters to match the music, rhythm and manner of motion are no doubt basic requirements.
Most likely the algorithm fails because musical rhythm involves a complex grammar that can only be understood by listening ahead and backward, by anticipation and memory: something we do all the time while listening to music, though mostly without knowing it. This musical grammar allows the musician to vary the durations, lengthening one beat, shortening another, without obscuring the metre. Visualizing music in real time may well be impossible except when it is most predictable.
Music visualization demos (such as these) often contain music that is rhythmically utterly primitive and rigid. Maybe contemporary pop music is suffering the effects of its visualization. The stuff that feeds your eyes in the end might starve your ears.