Herman Haverkort

I am exploring approaches to make sonifications or music of space-filling curves. My approach and possible implications are described in this paper (11 pages; this is a more detailed version of the paper that appears in the proceedings of Bridges 2017).

This page contains a number of sound tracks of space-filling curves. They are provided here as examples, interesting examples, but not necessarily good and well-polished examples. For a quick introduction, start with the following:

• the beginning of the Meander curve, played slowly so that it demonstrates most clearly what this is all about. Listen to the whole sound track if you like it, or skip to the next one if you get the point and prefer something faster:
• the world map curve;
• and now for the “real” stuff: listen to the musical texture of a five-dimensional Hilbert curve (version 1), in which you can hear rotations in five-dimensional space;

In addition to the above, the following material is available:

• A sonification of a four-dimensional curve (this is actually my personal favourite)
• Sonifications of three-dimensional curves;
• More two-dimensional curves, including a sonification of the Gosper flowsnake curve based on barycentric coordinates;
• Continuous-pitch sonifications (warning: not music, I guess);
• With a different technique (originally explored by the composer Tom Johnson for the Heighway dragon curve, with very nice results): the Hilbert curve through turning-function mapping;
• With a variation on this technique that is simplified and adapted to higher dimensions: renderings of the Harmonious Hilbert curves in up to six dimensions.

The approach and its background, practical implementation, and possible implications are described in this paper (11 pages; this is a more detailed version of the paper that appears in the proceedings of Bridges 2017).

The Meander curve as described by Wunderlich, 729 vertices in 10 minutes
without intro
with intro

The sound track plays the above sketch vertex by vertex, at a leisurely pace of 75 beats per minute, so it is quite possible to follow the curve every step of the way while listening. The sound track is a “prototype” in which supporting voices gradually join in with the primary voices playing the curve (dynamics and articulation are rudimentary). The vertices that are marked in the figure (at 1/4, 2/4, and 3/4 of the length) are also marked musically. For first hearing, take the track without intro, then you know when the curve starts.

If you would like to know what it sounds like without supporting voices, here is the “raw” sound track:

The following sample plays the Meander curve in a quite different style, and a lot faster. In this track, all notes stem from the space-filling curve, but in different ways. There are three pairs of voices: a melodic pair, a rhythmic pair, and a “coarser” pair. The melodic pair plays the 27 x 27 grid of 729 notes, similar to the track described above, but now in reverse direction, and with different scales. The rhythmic pair has the same pitches as the melodic pair, but staccato, with dynamics controlled by the curve as well: the beats are stronger in the four corner subsquares out of the nine subsquares of any square (and recursively so). The coarse pair plays the curve at a slower sampling rate: effectively each group of 9 beats is merged into one chord with the pitch values of the 9th beat. The rhythmic pair of voices plays throughout the whole track; each melodic voice plays through most of the track but is sometimes replaced by the corresponding coarser voice.

A curve that reminds me of a world map (126 vertices in 30 seconds)
	version 1 (shown)	version 2
raw sound track
processed

This curve was also described by Ventrella. The figure also shows the generator (in grey), and the pitch mapping for version 1. One clearly hears the strict syncopic structure of the counterpoint, caused by the fact that horizontal and vertical segments in the sketch of the curve alternate. The curve is played quite fast, but it is possible to follow what is happening if one focuses on one voice at a time. The vertical axis (horizontal segments) plays the bass, the horizontal axis (vertical segments) the top voice.

Version 2 plays the curve in the opposite direction, and with a different mapping (in which the horizontal keyboard is also reflected, so the track starts at the right end with the top voice medium high and the bass low, and ends at the left end with the top voice high and the bass low again.

Below are two renderings of Butz's five-dimensional Hilbert curve, sampled on a 4x4x4x4x4 grid. This is simply “raw” material, except for the addition of a gradual crescendo and decrescendo, no additional voices. The sound track covers the 1024 grid points along the curve in 3.5 minutes. The first version sounds fairly gentle, the second is more adventurous.

In both versions, we can hear several characteristic properties of the curve. Here I will highlight only some of these. The curve fills 32 subcubes of a five-dimensional cube one by one. This results in 32 musical “blocks” of about 6 seconds each. These differ in how they are rotated in five-dimensional space: we can hear this in the music because the rotation determines which voice has which rhythm. In fact, only five different rotations are used. The first and the last block share the same rotation, and all other blocks appear in pairs that have one of the four remaining rotations. As a result, we effectively hear a short first block (6 seconds), 15 long blocks (about 13 seconds each), and a short final block. Every odd long block has the same rotation. These blocks alternate with the other 7 long blocks, which use the remaining three rotations: one of these is used four times, one is used two times, and one is used only once (in the central block), clearly marking the middle of the sound track.

As described in my paper, there are theoretical considerations that suggest that three- and four-dimensional space-filling curves might have the greatest musical potential. Below is a first experiment in four dimensions: the “squared Hilbert curve” as described in my manuscript on higher-dimensional curves. Technically, it is obtained from the two-dimensional Hilbert curve as follows: consider the Hilbert curve as a mapping h from the unit interval to the unit square; now convert each two-dimensional point h(t) to a point with four coordinates by applying h to each coordinate of h(t). Thus we obtain a mapping from the unit interval to a four-dimensional cube, which turns out to have an interesting rhythm.

The sound track plays the 4096 points of a 8x8x8x8 grid in 14 minutes, simply the “raw” material, except for the automatic addition of dynamics. The dynamics are governed by the curve as follows: the music is softer when visiting points that are close to the starting point (by L1-distance in four-dimensional space), and louder when visiting points close to the second-last corner of the four-dimensional cube that is visited by the curve.

Here is an impression of the three-dimensional Meurthe curve as described here. It is 729 points of a 9 x 9 x 9 grid in 2 minutes, “raw” material, apart from some crescendo:

Below are three different three-dimensional Hilbert curves as described in my article (“raw” material, 4096 sample points in 14 minutes, ending abruptly). These tracks are unpolished (and probably too long), but it may be interesting to check out the rhythms.

Cl00.cf.ff.dd is a so-called standing curve: the similarity transformations between the curves within the octants and the curve as a whole keep the vertical axis vertical. This is clearly audible: two voices (the horizontal axes) imitate each other's rhythms, while the other voice (the vertical axis) has a (slow) rhythm of its own.

The Beta curve is hyperorthogonal, which implies that no note in any voice is shorter than 2 time units.

Below is an attempt to create a “largo” from a space-filling curve. To get long notes, I used a Meander-like curve on a 4×4 grid, and I sampled it at a resolution of 16×16 (see figure). Each sample point sounds for 1.1 second. The melodic line of the second voice seemed to call for jumping down by a seventh instead of moving up by a second in “strategic” places, hence the non-standard selection of pitches. In the figure, the jumps are indicated by crossbars. Two hand-composed supporting voices are added below the two voices determined by the curve.

Focusing on the technicalities of creating music from space-filling curves (and not on the obvious fact that I am not a trained composer), I am not quite satisfied with the result yet. Due to the space-filling character, all combinations of first-voice and second-voice pitch levels appear, and some of these are very dissonant (minor seconds). I did not manage to give all of these dissonances a proper place in the music. Possible solutions to explore:

• try other transpositions (maybe there is one where the dissonances are in less awkward places);
• try a hexatonic scale;
• try a curve that fills a triangle in a square grid (such as the Sierpinski curve) and position the triangle in the two-dimensional pitch space such that the second voice always remains at least a major second below the first;
• convince a trained and/or talented composer to have a go at it.

Using barycentric coordinates, we can also make a sound track of a curve that is drawn on a triangular grid, using three voices. Here is the raw material of a first attempt at the Gosper “flowsnake” curve. Theory suggests that two-dimensional curves may be boring—personally I believe that the sound tracks at the top of his page show that this is not always true. For the Gosper curve, I am undecided. Maybe this curve has musical potential, maybe not.

The well-known Gosper curve (344 vertices in 5 minutes)
raw sound track

Below are some examples of continuous-pitch renderings, 6 minutes each. This is unlike anything else on this page: there are no notes, the sonification is really a continuous mapping from time to pairs of pitch values. So, in a way, you could say that this is what space-filling curves really sound like:

generator and sketch	audio

Note: the first of these is a space-filling traversal, but not a space-filling curve, since it is not continuous.

The composer Tom Johnson wrote music based on the Heighway dragon curve, by encoding the left and right turns of a sketch of the curve as follows: at a left turn, move up in pitch; at a right turn, move down in pitch. This turns a traversal of the curve into a sound track that does not give the listener a sense of location the plane, but instead gives a sense of how much the curve spirals in and out while filling the plane. Because the sketch only defines one voice, it is relatively easy to overlay sketches of different refinement levels. (Johnson appears to do this in Formulas for String Quartet, part 7.) Below I apply such techniques to the Hilbert curve.

We sketch the path of the Hilbert curve through a grid of 4^k squares by drawing, in each square, a line segment from the point where the space-filling curve enters the square to the centre of the square, and from the centre of the square to the point where the space-filling curve leaves the square. In the sound track I overlay sketches for k = 1, 2, 3, 4, 5, played by five voices centred around G1, G2, G3, G4 and G5 respectively.

Only the first half of the curve is rendered. The coarser, and therefore shorter, sketches (k < 5) are spaced out with rests to remain synchronized with the most detailed sketch (k = 5). The sketches for k = 4 and k = 5 are truncated at the beginning and the end of the track, starting later and ending earlier than the other sketches. Phrasing is achieved by voices swapping sketches in a regular pattern, and by a regular progression of soft chords in the background (these are the only notes not determined by the space-filling curve). Dynamics are determined by location in the plane, getting louder when moving in the direction of the point half-way the rendered section of the curve, and softer when moving away from it.

Below I start from a sketch of the Hilbert curve through a grid of 4^k squares in the style of Hilbert himself: the sketch connects the centres of the squares in order. In his book Self-Similar Melodies, Johnson also proposes to use this style, and gives the melody (in abstract form, without a concrete scale) for k = 3. In my sound track I overlay sketches for k = 2, 3, 4, 5, 6, played by five voices centred on D2, A(!)3, D4, D5 and D6 respectively.

The top three voices use a pentatonic scale with D, E, F♯, A and B; the fourth voice uses a pentatonic scale with G, A, B, D, and E; the fifth (bottom) voice additionally includes C2 in the scale. The coarser sketches (k < 6) are stretched so that they synchronize with the most detailed sketch (k = 6). The sketch for k=6 is rendered at 7.5 notes per second, the sketch for k=2 has notes of more than half a minute. In the beginning of the sound track, the five voices enter one by one: this is because each sketch starts with an edge from the first square to the second, and it is only after half a minute that we reach the second square of the 16 squares represented in the coarsest sketch. The final chord represents a line segment that would connect the whole unit square to another square to the right of it.

Loudness is determined by distance (according to the L_∞-metric) to the end point. Changes in relative volume of the voices and instrumentation (decaying versus sustained notes) are used to emphasize or suppress sketches for certain values of k in certain sections of the sound track. The melodic “climax” is reached after approximately 7.5 minutes, when the finest sketch completes a 540 degrees' turn to the right, taking it up to E7; a few seconds after that, the coarsest sketch completes it 180 degrees' turn to the right to enter the 14th of square of 16. In some sections in the last two squares, some voices are amplified by high or low “bells”. In the final square, voices swap sketches among them, ultimately moving the finest and fastest sketch down to the bass voice, to obtain a satisfactory closing.

All notes in the sound track are space-filling curve notes; no hand-composed notes have been added. To get a pleasant sequence of harmonies, all I did is choosing the different scales for the different voices carefully.

For d = 1 up to 6, the following sound track plays, one after another, a sketch of how a d-dimensional Harmonious Hilbert curve (as described here) traverses a grid of 4^d subcubes. The track uses six different pitch classes (A, G, F, E, D, C) that correspond to steps in the 1st, 2nd, 3rd, 4th, 5th and 6th dimension, respectively (whether the step goes up or down in that dimension is not encoded). On top of that, there are 2d−1 voices that each sketch one of the 2d−1 facets of the d-dimensional cube that are each traversed in the order of the (d−1)-dimensional Harmonious Hilbert curve. In these sketches, more emphasis is given (by means of slower decay of the sound burst) to each 2nd, each 4th, each 8th, each 16th step etc. The sketches for the facets are aligned with the sketch of the d-dimensional curve, that is, the “steps” of the sketch of each (d−1)-dimensional curve are played when the d-dimensional curve makes the corresponding steps. Thus, the entire sound track is unisono, but the voices for the 2d−1 facets sound in different registers (the axis of the outward normal determines the octave, and the sign of the outward normal determines the timbre).

Note that in the six-dimensional section, the sketches for the facets play 11 times 1023 notes, while the six-dimensional sketch is only 4095 notes. Thus, on average, each note is hit by almost three facet voices. It is mostly the variations in how many (and which) facet voices are playing that create the dynamics of the sound track (there are no hand-composed notes or dynamics on this track!). One may listen to the salient features of the unexpectedly rich sound track and think about how those features arise from the structure of the curve and the six-dimensional cube—or one can just listen to the “music”.

Below is a track with an alternative pitch set (C, D, E♭, F♯, G), playing the Harmonious Hilbert curves up to five dimensions:

The following track plays the curve up to four dimensions (on C, B, A, G), but at a higher resolution (8 × 8 × 8 × 8), resulting in a track that is less dense: there are 7 facet voices of 511 notes each on a track of 4095 notes, so there is lots of “quiet time” in which one hears only the four-dimensional curve.