← The Director

The Director · notes

How the scores are made

Every sound in these scores is computed in Python.

The PROOFS film's score and the three made for THE DIRECTOR (THROUGHLINE, ENSEMBLE and CANON) are made the same way. A Python program computes every number in the sound file, one after another, from formulas, so nothing is recorded, sampled or run through a plugin. The two libraries it uses most are numpy (fast arithmetic on long lists of numbers) and scipy (filters). A third, pyloudnorm, only measures loudness at the end.

Halflight, also on this site, was made another way (section 6).

1

What a sound file is

A sound file is a list of numbers. For these scores it is 48,000 numbers per second for each ear, each between −1 and +1. A speaker cone moves out and in to follow them. That is all a sound is to a computer, so making music this way means writing the formula that fills the list.

The smallest complete example is two seconds of the note A:

import numpy as np
SR = 48000                              # samples per second
t = np.arange(2 * SR) / SR              # the time of every sample: 0, 1/48000, 2/48000, ...
a = 0.3 * np.sin(2 * np.pi * 440 * t)   # A440: a sine wave that repeats 440 times a second

Write a to a WAV file and you hear a pure tone. Everything else on this page builds on that last line.

2

The tools that do almost everything

Oscillators: pitch

A sine wave at frequency f is sin(2π f t). When the pitch has to move (vibrato, a glide, a tempo change), the program adds up the frequency sample by sample instead and takes the sine of the running total. That running total is the phase:

def phase_of(freq, n):
    return 2 * np.pi * np.cumsum(freq) / SR    # freq can change on every sample

Harmonics: colour

A violin and a flute playing the same A differ in their harmonics: the extra sines at 2, 3, 4 ... times the main frequency, and how loud each one is. Fourier showed that any repeating shape can be built from such a list, so a timbre is a recipe: a list of loudnesses, one per harmonic.

Harmonics come with one trap, called aliasing: any harmonic above half the sample rate (24 kHz here) folds back down as a wrong, sour note. Every oscillator in these scores drops those harmonics before it plays, which is called band-limiting.

Envelopes: how loudness moves

A struck object starts loud and dies away. Mathematically, that is multiplying by exp(−t / τ), where τ sets how long it rings. Every note also gets an attack of a millisecond or two, because a sound that starts instantly clicks. Slow fades use a smoothstep curve, 3x² − 2x³, which starts and ends gently.

Modes: what struck things sound like

A bar of metal or a bell has no single pitch. It rings at several frequencies at once, its modes, and physics fixes their ratios. For a free bar, the beam equation gives 1 : 2.756 : 5.404 : 8.933. A handbell's strongest partials sit near 1 : 3 : 4.16 : 5.43. Play those sines together, each decaying at its own rate, and you have the object. This is modal synthesis:

MALLET = ((1.0, 1.0, 1.0), (2.756, 0.26, 0.32), (5.404, 0.10, 0.12), (8.933, 0.03, 0.05))  # ratio, level, decay

def mallet(f, dur=2.4, decay=1.1):
    t = np.arange(int(dur * SR)) / SR
    y = sum(level * np.sin(2 * np.pi * f * ratio * t) * np.exp(-t / (decay * share)) for ratio, level, share in MALLET)
    return y * smooth(0, 0.002, t)       # a 2 ms attack, so it doesn't click

ENSEMBLE's players use this bar. CANON's bells use the handbell ratios. The PROOFS score's RESONANCE section used the Chladni plate's own frequencies.

Noise and filters: breath, sand, the cut

Random numbers sound like hiss. A filter keeps a band of that hiss and drops the rest, which turns it into wind, sand or the scratch of a burin. scipy's Butterworth filters do this in one call.

Space: the room

A room's reverberation is the original sound plus thousands of fading echoes. The program makes a fake room, about four seconds of noise that fades and gets darker as it fades, and convolves the dry sound with it. Convolution plays the sound once from every moment of that noise, scaled by the noise. It is the same mathematics a hall's acoustics perform, applied to an imaginary hall. Panning places each voice between the ears with an equal-power curve, so a voice doesn't get quieter as it crosses the middle.

Mastering: a fair loudness

Last, the whole mix is set to −16 LUFS, a standard measure of perceived loudness (the same target the PROOFS score used), and kept under −1 dB true peak. True peak means the loudest point between samples, which is what a speaker actually reaches.

3

How the scores are checked

Claude, which composed these scores, can't listen to them. So every score is measured, and the fixes come from the measurements:

  • Loudness over time in 5-second windows. This shows whether the piece has the arc it is meant to have.
  • Spectrograms: a picture of the sound, time across and pitch up, which shows the harmonics, the sections and anything odd.
  • Spectral balance (where the energy sits, from bass to treble), compared with the PROOFS score and Halflight.
  • True peak, DC offset and clicks: a click shows up as a jump between neighbouring samples far larger than the music around it.
  • The mathematics itself, measured in the output: how together ENSEMBLE's players actually are, and a check that CANON plays every pulse exactly once.

What the checks changed on the way:

The problemThe fix
Shawn saw a bullseye on the nose: the spiral's first turns sat on the brightest part of the faceThe spiral's centre moved just off the plate's corner. Every turn now crosses the face as one long curve, and THROUGHLINE was worked out again from the new line
THROUGHLINE dipped once per turn of the spiral: neighbouring wavetables partly cancelled when blendedEight wavetables per turn instead of one, so neighbours blend without cancelling
THROUGHLINE was too dark (spectral centroid 174 Hz first, 216 Hz after the new line; PROOFS sits at 280)The upper harmonics were turned up until its balance matched PROOFS: 339 Hz, a third of the energy below 120 Hz
The needle spent a long stretch in the dark backdrop, where the sound fell 12 dBThe needle now crosses the shirt in 3 seconds and the backdrop in 3, and spends 78 on the face
The ending fade started before the needle stopped and swallowed the final chordThe fade starts after the needle leaves the face, so the chord is heard
In the portrait's shader, thin lines came out about half as bright in a band along each axis (37 to 45 against 67 to 103, in linear light)Each pixel now shades both walls of the groove by how much of each it covers (found the same way, by measuring the picture)

The measurements catch mistakes, but they can't tell whether a piece moves you. That part is up to your ears.

4

The scores

PROOFS

Each section of the PROOFS film's score is synthesized from the mathematics of the piece it plays under.

  • RESONANCE: the Chladni plate's own frequencies, f = (196/34)(n² + m²), in the order the film shows them, over the hiss of sand.
  • APERIODIC: the Fibonacci word (L becomes LS, S becomes L), the one-dimensional cousin of the Penrose tiling. It plays long and short notes that never fall into a repeating pattern, pitched by the Beatty sequence ⌊nφ⌋.
  • LIMIT: a Shepard–Risset glissando, a tone that falls forever without getting lower, under a three-note chord for the three tiles that meet at every corner of the {7,3} tiling.
  • TEMPEST: the film's two Lorenz paths, started 10⁻⁹ apart, as two voices. They begin in unison, beat against each other, then become two different melodies.
  • MORPHOGEN: crackle whose density follows the reaction–diffusion growth front.
  • EPICYCLE: the AD-5 mark as sound. The film draws the mark as a sum of circles turning on circles (Fourier epicycles), and the score plays the same path, its x coordinate in the left channel and y in the right. Feed the two channels to an oscilloscope in XY mode and it draws the mark.

THROUGHLINE: the face, played like a record (1:34)

The idea. The portrait on the studio page is one spiral line, and the line's width is the photograph's light. The spiral's centre sits just off the plate's lower-left corner, so each turn crosses the plate once, as one long arc from edge to edge. Read one of those passes end to end and its width rises and falls with whatever it crosses: beard, mouth, eye, hair. That rise and fall is one cycle of a waveform. A wavetable synthesizer plays a stored cycle over and over at the pitch you ask for, so the face becomes the sound's colour.

The first version started the spiral at the tip of the nose, as Mellan's does. It put a bullseye on the nose, so the centre moved off the plate, and the score was worked out again from the new line.

The mathematics. The spiral is r = s(θ/2π + n): turn n, angle θ, s the distance between turns. For each pass the program reads 2,048 points of the line's width along the arc, from the same texture the portrait's shader draws with. It tapers the two ends so the cycle loops without a click, removes the average, keeps the harmonics each voice can carry without aliasing, and softens the top ones:

spec = np.fft.rfft(rows - rows.mean(axis=1, keepdims=True), axis=1)    # each pass, as a list of harmonics
tilt = np.where(keep, k.clip(1) ** -0.15 / (1 + (k * f0 / 5200.0) ** 2), 0.0)
tables = np.fft.irfft(spec * tilt, n=M, axis=1)                          # back to waveforms, one per pass

A needle follows the spiral on a schedule written as [seconds, turn] breakpoints: the dark shirt in 3 seconds (turn 16 to 90), then the face at a steady 1.51 turns a second for 78 seconds (turn 90 at the beard to 208 at the crown), then the backdrop in 3 seconds while the last chord rings. Every voice plays the pass the needle is on. When the portrait performs, it reads the same list, so the cut you see is where the needle is. The face's own contrast sets the volume: the beard and the eyes are loud and the dark shirt is nearly silent. The square root of the contrast is used, so the quieter passes can still be heard.

The layers. The groove is four voices per chord, each voice two copies of the wavetable played slightly out of tune with each other. The burin is filtered noise, as loud as the line is wide at the needle's exact point and heard only while the burin is on the plate. It crosses the plate once a turn, 255 times in all, which gives the piece its pulse, and in the performance each swish lands as the burin crosses the face on screen. The pedal is F♯1, under all six chords, as bright as the pass is light. It is the one note that never changes, the throughline in the title.

Chosen by the composer: the six chords (F♯ minor, D, B minor, E, C♯ suspended, then F♯ major at the crown), the needle's schedule, the tilt of the harmonics, the mix.

The engraved portrait with four passes of the line lit, through the beard, the mouth and nose, the eyes and the hairline, each beside the waveform it becomes
Four passes of the line are lit on the engraving (through the beard, the mouth and the nose, the eyes, the hairline), and beside each is the wave it becomes as a voice plays it. One pass, read end to end, is one cycle of the sound.

ENSEMBLE: many tempos become one pulse (1:44)

The idea. Thirty-two players each have a natural tempo of their own, spread around 96 BPM the way real people's are. Each plays a note every time it comes round. First they ignore each other, then they start to listen, and then a conductor arrives.

The mathematics. The Kuramoto model (1975), which also describes fireflies flashing together, pacemaker cells and audiences falling into rhythmic applause:

dθᵢ/dt = ωᵢ + (K/N) Σⱼ sin(θⱼ − θᵢ) + F sin(Ωt − θᵢ)

θi is where player i is in its beat, and ωi is its own tempo. K is how hard they listen to each other: each player is pulled toward the others' beats. F is the conductor, a beat at tempo Ω pulling everyone toward it. Kuramoto's result is that a common pulse appears suddenly, once K passes a critical value set only by how spread out the tempos are. For this spread it is Kc = 2/(πg(0)) = 0.50. The program integrates the equation 2,000 times a second and writes down the exact moment each player's phase comes round. Each of those moments is a note.

How together they are is the order parameter, r = |(1/N) Σj eiθj|. It runs from 0 (scattered) to 1 (one pulse). The chord underneath is played at a loudness of r², so the harmony only sounds when they agree. Measured from the run:

Sectionr
The cloud, before they listen (0:02 to 0:14)0.14
Listening, past the critical coupling (0:36 to 0:44)0.92
Conducted, 96 rising to 108 BPM (0:52 to 1:10)0.99
The conductor gone, still listening (1:16 to 1:24)0.94
Not listening any more (1:38 to 1:42)0.05

When the conductor leaves, the players hold together, and their shared tempo slides back from 108 to their own average of 96. The run measured 96.0 BPM while they listen, 108.0 under the conductor and 96.0 again once it has gone. The slowdown comes from the equation itself.

Chosen by the composer: the tempo spread, the schedule of K and F, the conductor's tempos, the five-note sets each section uses, the mallet's sound.

Seven wheels of the players' phases, scattered then gathered into one point then scattered again, above the curve of r over time
Seven wheels show every player's phase as a dot at seven moments, scattered, then gathering, then one point, then scattered again. Under them is the curve of r with K and F.

CANON: every beat played exactly once (1:29)

The idea. Six players each have one bell and play the same rhythm, each entering at a different point. Together they ring every pulse exactly once: no pulse is skipped, and until the final cue no two players strike at once. Each player's own part sounds irregular, and the melody exists only in the hand-offs between them.

The mathematics. A rhythmic canon tiles time when one rhythm A, entered at offsets B, covers every pulse once. Written A ⊕ B = ℤn: every number from 0 to n−1 is a + b (mod n) in exactly one way. Dan Tudor Vuza (1991) found the rare kind where neither the rhythm nor the entries repeat inside the cycle. The smallest exist at n = 72:

A = {0, 1, 5, 6, 12, 25, 29, 36, 42, 48, 49, 53}     the rhythm: 12 strokes in 72 pulses
B = {0, 8, 16, 18, 26, 34}                           the six entries

The script checks this before it plays a note. If it were wrong, the program would stop:

sums = sorted((a + b) % 72 for a in A for b in B)
assert sums == list(range(72))                        # every pulse, exactly once
assert not periodic(A) and not periodic(B)             # and neither part repeats: a Vuza canon

The form. The players enter one per 72-pulse cycle. Once the sixth has entered, every sixteenth note is filled. Two more cycles change the chord under the same hand-offs, each player moving their bell to the new chord. The only collision in the piece is the last moment, when all six strike together on the cue.

Chosen by the composer: the tempo (112 BPM), the bells' pitches and chords, the bell's partials, the drone.

A grid of six rows and seventy-two columns; each column has exactly one filled cell; the bottom row is filled in every column
The grid has six rows, the players, and 72 columns, the pulses. Each column holds exactly one stroke, and the bottom row, which combines all six, is full.

5

What is computed and what is chosen

Computed by the mathematicsChosen by the composer
THROUGHLINEEvery timbre, every swell, the burin's rhythm, the order of events (the face, beard to crown)Six chords, the needle's schedule, how bright, the mix
ENSEMBLEEvery note's timing; when the pulse appears and dissolves; the drift back to 96 BPM; how loud the harmony isTempo spread, when they listen, the conductor's moves, the pitch sets
CANONWhich player plays every pulse, so the whole melody's shapeTempo, the bells' notes and chords, the ending
PROOFSPitches and rhythms from each equation; the oscilloscope drawingOrder, levels, the bed under it

The mathematics supplies structure, timing and colour, and the composer chooses the rest, the harmony above all. That split is deliberate, and each script's opening comment says which is which.

6

Halflight is different

Halflight was also composed as code in Python: its notes, chords and drum patterns, and its mix and master. Its sound design used studio tools, though: the Valhalla reverbs and delay as plugins (driven from Python through Spotify's pedalboard library), a Prophet-style synthesizer written in numba, and some Logic one-shots, and it was assembled in Logic Pro. It is a produced track, an original song written in the style of Rival Consoles, with no affiliation to the artist. The PROOFS score and these three are pure synthesis from equations.

7

One script per score

THROUGHLINE, ENSEMBLE and CANON are one Python script each, with a small shared library of synthesis and measurement code. A run writes the sound as a 48 kHz, 24-bit WAV, a spectrogram, the picture that checks its mathematics and a file of measurements, in a few seconds to half a minute. Revising a score means changing a number at the top of its script (THROUGHLINE's needle schedule, ENSEMBLE's tempo spread, CANON's tempo or bells) and running it again.

8

Where this could go

For now the scores and the pictures are made separately. They meet in a recording and a short list of timings: THROUGHLINE plays as a sound file, and the portrait follows the needle's schedule against it. One next step would make them one thing in the browser: the same equation stepping the image on the graphics card (WGSL) and the sound in an AudioWorklet, so a visitor's hand changes both at once. ENSEMBLE is the natural first candidate. Its 32 players could be 32 points of light that a visitor conducts.