(Björk, NYC, 2007)
(ROTOR by ReacTable)
We met Sergi Jordà on a cold and sunny winter day last March in Barcelona. And while we shared thoughts about common interests, careers and music he unfolded his great personality: affable, warm, and yet incredibly smart and outgoing. He is the kind of person you imagine has a brain working at a zillion rpm per second. Raised in a creative family – his father was the award-winning Catalan film director and writer Joaquín Jordà – this innovator, installation artist, digital musician has been developing a more than impressive career. He is also an Associate Professor at the Music Technology Group at Universitat Pompeu Fabra in Barcelona.
Sergi is best known for directing the team that invented the ReacTable. Even though he’s a trained physicist and musician, it was only when he discovered computers and programming he knew what that he’d found what it would become his biggest passion. His professional interests have been focused for a long time in musical interfaces, and more particularly in haptics interfaces. Haptics refers to a type of human-computer interaction technology that encompasses tactile feedback or other bodily sensations to perform actions or processes on a computing device. And so, Haptic interfaces are systems that allow a human being to interact with a computer through bodily sensations and movements. With this concept in mind, Sergi envisaged the ReacTable.
ReacTable is a tangible, electro-acoustic musical instrument that allows collaborative live performances without the limitation of screen-based interfaces. The device was conceived and it’s design started in 2003 by Sergi Jordà, Günter Geiger, Martin Kaltenbrunner, and Marcos Alonso at the Pompeu Fabra University. The team presented their creation for the first time in a public concert at the International Computer Music Conference 2005 in Barcelona.
The ReacTable became broadly popular on the internet, particularly in the ‘blogosphere’, to the extent the world-acclaimed Icelandic singer-songwriter Björk incorporated the Reactable on her world tour Volta (2007-2008) (she continued to use the instrument during the Biophilia tour (2011-2013)). ReacTable also has received several prestigious international prizes and awards such as the Prix Ars Electronica Golden Nica for Digital Musics in 2008.
After ReacTable, the team has endeavoured to adapt to developing mobile technologies. This last November they presented a new invention, ROTOR, which is an app that turns the iPad into an electronic music performance suite. Using the optional ROTOR controllers, a kind of particular physical control objects, ROTOR brings the tangible experience of the ReacTable for the first time to a personal platform like the iPad.
Work on ReacTable and ROTOR aside, Jordà’s has had a prolific academic career. He has become one of the world leading experts in haptics interaction with an extensive publication list and a thriving research group. In times where new digital interfaces seem to blur the boundaries between the physical and the digital, one of the key questions to answer is ‘how do people respond to multiple stimuli in a digital environment beyond the touch of a screen or button?’ We hope the work of Sergi Jordà keeps resonating as inspiring as it already has been.
Words by Meritxell Rosell (Twitter @dancingmoog)
Your work is at the intersection of technology and sound, when and how did the fascination with them come about?
I believe it started when I was 14. As a matter of fact, I don’t think I had been much interested in music before that, although I have always been interested in (low) technology and, as a kid, I enjoyed opening broken devices for trying to fix or reconceiving them… giving them new incarnations… sort of circuit bending (1), I guess.
Then, in 1975, music entered my life, and it did it in a very vehement way. I started to listen to all types of music compulsively, although my first favourite bands at that time were King Crimson, Soft Machine and Van der Graf Generator. But unlike many teenagers, I didn’t feel like picking a guitar. Rather, I remember, for example, collecting old mikes and speakers from small radios, to build a sort of talk-box (2), with which I would process some of my favourite solos on records. I especially remember working with some John Coltrane’s early 60s albums on Atlantic, because the stereo mix was completely separated, so I could process Coltrane’s saxophone through my mouth, without affecting the bass or the drums! Another day, I started recording and collecting all types of “busy” and “not answering” telephone sounds in cassette tapes. I wanted to use them for recreating live, using a bunch of cassette players, a minimal repetitive piece that was fascinating me (and still continues to), “The Soft Weed Factor” (3), from Soft Machine’s Sixth album.
Then in 1979 I entered University and started studying Physics. Although I shortly discovered that Physics was not the fun I had initially expected, a side effect was that at the faculty I also found computers and computer programming, and I soon realised that this was going to be my thing! At that time, I was also playing the saxophone a, sort of free-jazz, but I wasn’t too good at it. I quite consciously knew in fact, that I would never become a great saxophone player because I didn’t like to practice or rehearse. I like to understand processes, and perhaps do things a couple of times, but once I understand something, I prefer to pass onto something else, which is clearly not the best path for mastering any psychomotor activity in general, and any music instrument in particular!
So, one day, probably around 1982, while listening to Laurie Anderson’s “Oh Superman” (4) I realised that computers could be used for making music! Best of all, I already knew that computers don’t get tired of doing repetitive tasks, so that day I understood that, if told properly, computers were far better suited than me for repetitive and unexciting tasks. Computers learn whatever at the first take, and they can repeat it forever without a glitch, I thought. So I finally dropped practising the scales on the saxophone and started programming, hoping that, in a not too distant future, I would be able to improvise “my” own music with those machines. This is what, to some extent and with different degrees of success, I have been doing since then.
Reactable is your world acclaimed invention, popularised by Björk, who incorporated it into her 2007 Volta Tour. Could you tell us a little bit about the intellectual process that drove you to its creation?
The idea of Reactable came in a sleepless night, around 2003. By then, I had already spent at least 15 years programming interactive music and designing software/hardware musical devices. In some of them, notably in FMOL (5), which I developed in 1997 for La Fura dels Baus (6), you can already see a lot of Reactable’s ideas. In this period, I was also working on my PhD thesis which was precisely devoted to what I called “Digital Lutherie” (7)… So, while things sometimes seem to come out of the blue, I believe that what happens is that at one point, they emerge enough so that they just become visible to oneself.
Anyway, let’s say that for a bunch of reasons, the circumstances were favourable in 2003 for the Reactable to be imagined. This sleepless night I worked on most of its main concepts and a few days later I managed to convince a team of PhD students and amazing audio developers to join me on that project! These were Günter Geiger and Martin Kaltenbrunner, later to be joined by Marcos Alonso (8). We worked intensively on it during the following years, developing, often inventing the technologies needed for its completion. I have always said that Reactable started from a concept and a wish, and never from technology. Technology entered later. I first imagined what I wanted to build, and then we kept solving one by one the practical issues on how to achieve this.
In 2005 we presented the first prototype; in late 2006 we published a video on YouTube which became unexpectedly popular (9); in 2007 Björk contacted us for using the Reactable on her Volta Tour (10), and in 2009 we founded the company Reactable Systems (11).
What have the biggest challenges you faced in the development of new projects around Reactable? Can you tell us a bit about them?
When we started in 2003, we were a small team of only four researchers. In a way, pressure came only from within ourselves. Today things are different. Since January 2016 I have become Reactable System’s CEO. When you ran a company, especially a small and innovative one, pressure comes from everywhere, and all resources, especially human, never seem enough. For years, we have been improving our tabletops both from a software and a hardware perspective; we launched Reactable Mobile, which brought most of Reactable’s interaction concepts (i.e. minus the tangibles) to tablets and smartphones. We were extremely busy, and yet we were not producing unseen or unheard novelties anymore.
Luckily, for the last years, we have also been able to carry on some more “silent” research, and we have just started to launch new products. The first one, ROTOR, which has just been released in October is a completely revamped application specially designed for the iPad and the iPad Pro, which, using two small controllers, finally brings tangible control to tablets (12,13). We are very excited about ROTOR, and it is the first of more new products that will hit the market in the next months.
What direction do you see taking your work into?
During the last two or three years, at Reactable we have also been carrying extensive research in the field of “intelligent music instruments”, or instruments that “know” about music and about what they are playing. This research has been especially carried within the GiantSteps EC-founded project (14) in which we have collaborated with other European research groups, such as the Music Technology Group from Barcelona’s, the Jonathan Kepler University in Linz, STEIM in Amsterdam, and other music tech companies such as Native Instruments. Although GiantSteps has just ended this October, we plan to start releasing “stylistic music agents” in different software forms.
What keeps you up at night?
For years, I used to work mostly at night, but not anymore. I now prefer to go to bed earlier. And I am a very fast sleeper; I jump into bed and fall asleep within 5 minutes! However, when this doesn’t happen or, as it sometimes occurs when I suddenly wake up at 3 AM, it tends to be for good! That’s when my clearer and most disruptive ideas tend to arise. I pick my notebook (white paper, always without stripes or squares… a must) and a pen (always black liquid ink… another must) and I keep noting, drafting, formulating… for the whole night. New ideas, drafts, schemes, even cheap philosophy… I do often find hard to follow them in the morning, but I guess that this represents for me the clearest incarnation of such abstract and hazy concepts as “creativity” or “inspiration” …
You couldn’t live without…
I believe I am quite austere when talking about material needs. I am not a tech-freak at all, neither a fashion or consumer victim. I’m passionate about gastronomy, but I especially love to cook and experiment with food, so I don’t need to go to fancy restaurants every week either. But I am still extremely curious about almost anything, so for me, the first and essential purpose of the Internet is for learning and discovering. Our modern “library of Alexandria”… this is what I would probably first miss in my “desert island”. I guess I would also find quite sad to live without music and my beloved ones! And then, of course, I will always cherish a white notebook and a black liquid ink pen!
Websites: www.dtic.upf.edu/~sergi/ and www.reactable.com
(Photos courtesy of the artist)
Further readings: (1) https://en.wikipedia.org/wiki/Circuit_bending; (2) https://en.wikipedia.org/wiki/Talk_box; (3) https://www.youtube.com/watch?v=dU8n2HgT9B4; (4) https://www.youtube.com/watch?v=Vkfpi2H8tOE; (5) http://www.dtic.upf.edu/~sergi/?/mediaprojects/fmol/; (6) http://www.lafura.com/en; (7) http://mtg.upf.edu/node/449; (8) http://reactable.com/about-us/; (9) https://www.youtube.com/watch?v=0h-RhyopUmc; (10) https://www.youtube.com/watch?v=JFFh8Rtb_-E; (11) http://reactable.com/; (12) http://reactable.com/rotor/; (13) https://www.youtube.com/watch?v=oohDErI7Eg0; (14) http://www.giantsteps-project.eu/