Interview by Meritxell Rosell
We met Sergi Jordà on a cold, sunny winter day in Barcelona last March. And while we shared thoughts about common interests, careers and music, he unfolded his great personality: affable, warm, and yet incredibly smart and outgoing. He is the kind of person you imagine has a brain working at a zillion rpm per second.
Raised in a creative family – his father was the award-winning Catalan film director and writer Joaquín Jordà – this innovator, installation artist, and digital musician has been developing a more than impressive career. He is also an Associate Professor at the Music Technology Group at Universitat Pompeu Fabra in Barcelona.
Sergi is best known for directing the team that invented the ReacTable. Even though he’s a trained physicist and musician, it was only when he discovered computers and programming he knew that he’d found what would become his biggest passion. His professional interests have been focused for a long time on musical interfaces, more particularly in haptics interfaces.
Haptics refers to human-computer interaction technology encompassing tactile feedback or other bodily sensations to perform actions or processes on a computing device. And so, Haptic interfaces are systems that allow a human being to interact with a computer through bodily sensations and movements. With this concept in mind, Sergi envisaged the ReacTable.
ReacTable is a tangible, electro-acoustic musical instrument that allows collaborative live performances without the limitation of screen-based interfaces. The device was conceived and designed in 2003 by Sergi Jordà, Günter Geiger, Martin Kaltenbrunner, and Marcos Alonso at Pompeu Fabra University. The team presented their creation for the first time in a public concert at the International Computer Music Conference 2005 in Barcelona.
The ReacTable became broadly popular on the internet, particularly in the ‘blogosphere’, to the extent that the world-acclaimed Icelandic singer-songwriter Björk incorporated the Reactable on her world tour Volta (2007-2008) (she continued to use the instrument during the Biophilia tour (2011-2013)). ReacTable also received several prestigious international prizes and awards, such as the Prix Ars Electronica Golden Nica for Digital Musics in 2008.
After ReacTable, the team has endeavoured to adapt to developing mobile technologies. Last November, they presented a new invention, ROTOR, which is an app that turns the iPad into an electronic music performance suite. Using the optional ROTOR controllers, a kind of particular physical control object, ROTOR brings the tangible experience of the ReacTable for the first time to a personal platform like the iPad.
Work on ReacTable and ROTOR aside, Jordà has had a prolific academic career. He has become one of the world’s leading experts in haptics interaction, with an extensive publication list and a thriving research group. When new digital interfaces seem to blur the boundaries between the physical and the digital, one of the key questions to answer is, How do people respond to multiple stimuli in a digital environment beyond the touch of a screen or button? We hope the work of Sergi Jordà keeps resonating as inspiring as it already has been.
Your work is at the intersection of technology and sound; when and how did the fascination with them come about?
I believe it started when I was 14. As a matter of fact, I don’t think I had been much interested in music before that, although I have always been interested in (low) technology and, as a kid, I enjoyed opening broken devices to try to fix or reconceiving them… giving them new incarnations… sort of circuit bending [1], I guess.
Then, in 1975, music entered my life, and it did it in a very vehement way. I started to listen to all types of music compulsively, although my first favourite bands then were King Crimson, Soft Machine and Van der Graf Generator. But unlike many teenagers, I didn’t feel like picking a guitar. Rather, I remember, for example, collecting old mikes and speakers from small radios to build a sort of talk-box [2], with which I would process some of my favourite solos on records.
I especially remember working with some of John Coltrane’s early 60s albums on Atlantic because the stereo mix was completely separated. I could process Coltrane’s saxophone through my mouth without affecting the bass or the drums! Another day, I started recording and collecting all types of “busy” and “not answering” telephone sounds in cassette tapes.
I wanted to use them for recreating live, using a bunch of cassette players, a minimal repetitive piece that was fascinating to me (and still continues to), “The Soft Weed Factor” [3], from Soft Machine’s Sixth album.
Then in 1979, I entered University and started studying Physics. Although I shortly discovered that Physics was not the fun I had initially expected, a side effect was that I also found computers and computer programming at the faculty. I soon realised that this would be my thing! At that time, I was also playing the saxophone, a sort of free jazz, but I wasn’t too good at it.
I knew I would never become a great saxophone player because I disliked practising or rehearsing. I like to understand processes and perhaps do things a couple of times. Still, once I understand something, I prefer to pass on something else, which is clearly not the best path for mastering any psychomotor activity in general, and any musical instrument in particular!
So, one day, probably around 1982, while listening to Laurie Anderson’s “Oh Superman” [4], I realised that computers could be used for making music! Best of all, I already knew that computers don’t get tired of doing repetitive tasks, so that day, I understood that, if told properly, computers were far better suited than me for repetitive and unexciting tasks.
Computers learn whatever at the first take, and they can repeat it forever without a glitch, I thought. So I finally dropped practising the scales on the saxophone and started programming, hoping that, in a not-too-distant future, I could improvise “my” own music with those machines. I have been doing this to some extent and with different degrees of success since then.
Reactable is your world-acclaimed invention, popularised by Björk, who incorporated it into her 2007 Volta Tour. Could you tell us briefly about the intellectual process that drove you to its creation?
The idea of Reactable came on a sleepless night around 2003. By then, I had already spent at least 15 years programming interactive music and designing software/hardware musical devices. In some of them, notably in FMOL [5], which I developed in 1997 for La Fura dels Baus [6], you can already see many of Reactable’s ideas.
During this period, I was also working on my PhD thesis, which was precisely devoted to what I called “Digital Lutherie” [7]… So, while things sometimes seem to come out of the blue, I believe that what happens is that, at one point, they emerge enough so that they just become visible to oneself.
Anyway, let’s say that for many reasons, the circumstances were favourable in 2003 for the Reactable to be imagined. This sleepless night I worked on most of its main concepts, and a few days later, I managed to convince a team of PhD students and amazing audio developers to join me on that project!
These were Günter Geiger and Martin Kaltenbrunner, later to be joined by Marcos Alonso (8). We worked intensively on it during the following years, developing, often inventing, the technologies needed for its completion. I have always said that Reactable started from a concept and a wish and never from technology. Technology entered later. I first imagined what I wanted to build, and then we kept solving the practical issues on how to achieve this one by one.
In 2005 we presented the first prototype; in late 2006, we published a video on YouTube which became unexpectedly popular [9]; in 2007, Björk contacted us to use the Reactable on her Volta Tour [10], and in 2009 we founded the company Reactable Systems [11].
What were the biggest challenges you faced in developing new projects around Reactable? Can you tell us a bit about them?
When we started in 2003, we were a small team of only four researchers. In a way, pressure came only from within ourselves. Today things are different. Since January 2016, I have become Reactable System’s CEO. When you ran a company, especially a small and innovative one, pressure comes from everywhere, and all resources, especially humans, never seem enough.
For years, we have been improving our tabletops both from a software and a hardware perspective; we launched Reactable Mobile, which brought most of Reactable’s interaction concepts (i.e. minus the tangibles) to tablets and smartphones. We were extremely busy, yet we were no longer producing unseen or unheard novelties.
Luckily, for the last few years, we have also been able to carry on some more “silent” research, and we have just started to launch new products. The first one, ROTOR, which was released in October, is a completely revamped application specially designed for the iPad and the iPad Pro, which, using two small controllers, finally brings tangible control to tablets [12,13]. We are very excited about ROTOR; it is the first of more new products to hit the market in the next months.
What direction do you see taking your work into?
During the last two or three years, at Reactable, we have also been conducting extensive research in the field of “intelligent music instruments”, or instruments that “know” about music and what they are playing.
This research has been especially carried out within the GiantSteps EC-founded project [14] in which we have collaborated with other European research groups, such as the Music Technology Group from Barcelona, the Jonathan Kepler University in Linz, STEIM in Amsterdam, and other music tech companies such as Native Instruments. Although GiantSteps has just ended this October, we plan to start releasing “stylistic music agents” in different software forms.
What keeps you up at night?
I used to work mostly at night for years, but not anymore. I now prefer to go to bed earlier. And I am a very fast sleeper; I jump into bed and fall asleep within 5 minutes! However, when this doesn’t happen or, as it sometimes occurs when I suddenly wake up at 3 AM, it tends to be for good! That’s when my clearer and most disruptive ideas tend to arise.
I pick my notebook (white paper, always without stripes or squares… a must) and a pen (always black liquid ink… another must), and I keep noting, drafting, and formulating… for the whole night. New ideas, drafts, schemes, even cheap philosophy… I often find it hard to follow them in the morning, but I guess this represents for me the clearest incarnation of such abstract and hazy concepts as “creativity” or “inspiration” …
You couldn’t live without…
I believe I am quite austere when talking about material needs. I am neither a tech freak nor a fashion or consumer victim. I’m passionate about gastronomy, but I especially love cooking and experimenting with food, so I don’t need to go to fancy restaurants every week.
But I am still extremely curious about almost anything, so for me, the first and essential purpose of the Internet is for learning and discovering. Our modern “library of Alexandria”… is what I would probably first miss in my “desert island”. I guess I would also find it quite sad to live without music and my beloved ones! And then, of course, I will always cherish a white notebook and a black liquid ink pen!
