Text by CLOT Magazine
Lorem is a music-driven multidisciplinary project by the Italian musician and visual artist Francesco D’Abbraccio. Lorem is an inquiry about human-computer interaction in the age of Artificial Intelligence. During the last two years, D’Abbraccio has built a collaborative platform, involving AI and video artists, IT researchers and music instruments designers such as Mario Klingemann with the aim to produce original audio, visuals, and lyrics. Deep neural networks, hacked hardware, and programming errors are used to drive machines toward aesthetical, autonomous behaviours.
We are presenting a new video produced by AI art pioneer Mario Klingemann for Lorem’s upcoming album Adversarial Feelings. For Kilingemann, who was awarded a Lumen Gold Prize 2018 and is a Google Arts and Culture resident, his preferred tools are neural networks, code and algorithms.
Different kinds of Neural Networks (NN) were involved in generating lyrics, MIDI and video for Adversarial feelings, which consist of 9 AI-generated A/V pieces. The video was generated by Mario Klingemann with a couple of Generative Adversarial Networks (GANs), while audio and texts by Lorem with a Recurrent Neural Network and Long Short Memory Term (LSMT) Neural Network, respectively. The whole process was generated by artificial intelligence, and the project aims to teach AI to express human and non-human feelings. Francesco told us that the original drums by Lorem were rearranged by the Neural Network, which was converted into an audio stream. Then Klingemann used them to interfere in the process of two GANs (Generative Adversarial Networks) so the music could interact with the neural image generation process: Klingemann analysed the song and extracted the frequency spectrum and the onsets to use that data to directly influence the output of two GANs that he had trained for portrait generation.
Klingemann, whose artistic interests involve artificial intelligence, deep learning, and glitch art, hooked the frequency data directly up to the learned weights inside the GANs to glitch them. When weights are manipulated, even small changes can create “wrong” results; in the video, the music itself, the small change that directly glitches what the model has learned, and a similar combination of frequencies will result in similar errors.
The Sky would clear what the Man had wrapped to the Link is a collaboration by Lorem and Mario Klingemann ahead of Lorem’s debut album out on April 19.
