With the help of Berlin’s Humboldt University, the Stasi wanted to develop a system that could analyse sound by visualising it through a spectrogram or voiceprint. The aim of forensic acoustics was quite simply to identify an anonymous caller by matching an incriminating recording with an official one. “The Stasi soon found out how difficult it was to identify people. The voiceprint can differ depending on recording equipment, the quality of the connection, how nervous you are, your age, context, etc.”

The word choice is interesting too: “The term voiceprint implies a much greater power and accuracy when it comes to identifying people than the technology could actually provide.” The Stasi tried to resolve this by combining the logic of mass surveillance – gathering as much data as possible – with a relatively old school manual approach: listening to recordings and describing them. However, there were two big problems with this.

“It quickly emerged that perceptions of voices are very subjective: what is a deep voice? When would you describe someone as tending towards monologues?” The Stasi transitioned from free text descriptions to ticking boxes, to a semantic differential, i.e. a binary scale.  “They also developed an increasingly long, constantly evolving list of characteristics. The files show that there was plenty of disagreement on the best way to proceed.”

Too much data for the humble audio cassette

The other problem is the sheer volume of data required for a pre-emptive mass surveillance approach. Corollaries include serious epistemological and ethical questions – and archivists being treated to ten-second blasts of German Schlager in between recordings of interrogations or wire taps. “The Stasi regularly ran out of tape: not only did they have to reuse it, they also started confiscating audio cassettes from cars crossing the border.”

Probably more pertinent is the classification problem that comes with vast amounts of data. “What do you keep? What is it relevant for and when? More data actually makes the system slower; and the more categories you devise and change over time, the more difficult the retrieval issues.” Bijsterveld thinks the epistemological doubt about how to collect and store data so that it can be meaningful at the right moment is very relevant to big data today.

“Obviously, they have been responsible for atrocious things. It is weird to witness the hard work and dedication.” Dedication up to a point. “In November 1989, the logbooks stop and you find handwritten calculations of pensions…” The archives also reveal a surprisingly human side of the dutiful East German agents. “In a logbook, I found a poem probably obtained during a house search. I don’t know if they thought it was incriminating evidence or just a good poem…”

Unfortunately, she couldn’t go all the way in exploring this human side. “I was very excited when I got the contact of a scientist who had been working on the sound analysis technology.” But then Bijsterveld made an error she always warns her students against. “I thought I’ll wait until I have more information, lest I ask silly questions. When I came back, another researcher had already contacted the scientist. Apparently their conversation went badly and he was no longer interested in talking about the past.”

The world’s most accessible secrets

Bijsterveld started this line of research as early as 2009 but it took her a long time to obtain the necessary files. “Several years after I’d put in the request, I received a huge box with about a thousand copies of files – it takes so long because the names of all the subjects have to be blacked out.” That makes the Stasi archives unique: despite the understandable attention to privacy, this is the most accessible secret service in the world.

This is only her latest foray into researching auditory heritage. She started out tracking the history of noise as a public problem and has organised an installation on past soundscapes of Amsterdam, including interactive simulations of the Dam Square in 1895 and 1935, using recordings of historical artefacts based on a painting. “It’s fascinating but, of course, we now listen in a completely different way; we give different meanings to sounds.”

Understanding listening skills

Her main interest today is sonic skills, i.e. why and how people listen in different fields of science. “Listening is often contested more than seeing, which is why the Stasi initially relied on spectrograms, i.e. visual representation of frequencies.” Intriguingly enough, there also seems to be tendency in the opposite direction, as her colleague Alexandra Supper showed.

“Sonification means turning data into sound. Some claim it’s easier to detect patterns that way but this is still contested... There’s a playful side to it, too: CERN’s website shows some of the scientists playing their data on musical instruments.” There’s no claim of pedagogic efficacy here; particle physics is not more readily understood when interpreted by a bassoon, “but it gives some people the sense of being closer to this immensely abstract research.” Like listening to the music of the spheres…

That Bijsterveld lost her hearing in one ear a few years ago is both tragic and ironic – but not enough to slow her down. “I joke about it with my colleagues – in any case, a lot of my sources about sounds are actually visual...”

All information about Prof Bijsterveld’s current research were obtained legally and with her full consent.