Q&A

What is Automation?

AUTOMATION is a Cinematic Concerto that explores the conflicts and connections between man and machine. Watch and listen as cellist Yves Dhar and his holographic AI counterpart interact on stage with 2022 Musical America Conductor of the Year Teddy Abrams and the Louisville Orchestra in a groundbreaking new composition by Grammy-nominated, Emmy-winning composer Adam Schoenberg that features cello, halldorophone, electronics, orchestra and projected visuals. Can a technically superior yet emotionless machine create music that moves us?  How will humans react when faced with the possibility of being replaced? These are a few of the questions that this probing work intends to ask. A vision of a haunting future imagined in an adapted soloist + orchestra format, Automation promises to be a visceral experience that will leave audiences talking about what it truly means to be human.

I just watched the video. Looks cool!  But I have so many questions …

Excellent!  We’re glad we piqued your curiosity. Yves Dhar (Cellist and Producer) and Adam Schoenberg (Composer) are on hand to answer your questions below.

The promotional video looks like a trailer for a sci-fi film. Is Automation a movie?

Yves: — I’ve coined Automation as a “cinematic concerto,” or a “cine-con” if you’re into the whole brevity thing. That’s probably just as confusing, eh?  First and foremost, it’s a concerto, which is a traditional work of classical music that features one or more soloists accompanied by an orchestra. Bach, Mozart, Beethoven, Tchaikovsky all wrote many of them to showcase the star musicians of their days (or sometimes even themselves). Adam and I decided we wanted to push the concerto into the 21st century and bring in some film-like elements. Visual projections on a screen. Lighting. Electronics and Sound FX to mix in with the acoustic orchestra. So, no, Automation isn’t really a film, but since it has some stuff like one, we’re calling it a “cinematic concerto” that you experience in the concert hall instead of the movie theater.

So is that where the “holographic AI counterpart” comes in?  Is it going to be like Tupac at Coachella back in 2012?

Yves: — Yes and, um … maybe?  Automation is about man and machine, humans and artificial intelligence. So why not have both human and AI performers live on stage to represent themselves?  Well, we’re not quite there yet tech-wise in terms of having a true AI cellist perform live (or if we are, I wouldn’t know how to access it), so we thought to do the next best thing: project a life-like digital hologram to play its own part onstage instead.

Adam: — The hologram goes back to our first conversations about the concerto. When Yves came to me 4 or 5 years ago, his initial thought was that he wanted a piece that was relevant, timely, and could attract a younger crowd. So we immediately thought of tech and the world that we live in and what that means for a musical performance. And so came the idea of a concerto that is rooted in analog, AKA acoustic instruments, but putting it in the 21st century. We’re not digital natives, but we now live in a digital world.

So how does the hologram work?

Yves: — Oof. A magician never reveals his secrets!  Well, that tech in itself has evolved even over the 5 years that Automation was being developed. Holograms are nothing new. We think of them as futuristic, but they’ve actually been around since the 1800s. I remember seeing them as a kid in the Haunted Mansion at Disney World. Sure enough, when I researched that and how Tupac came back from the dead to perform at Coachella, they used the same basic technique: Pepper’s Ghost. I won’t go into the specifics of how that works, it involves mirrors and bounce screens, and calculated angles. It was safe to say it was way too difficult to set up – it wouldn’t travel well – and definitely way too expensive.

I was disappointed to find that out, but wouldn’t give up that easily. Down the rabbit hole I went, and I saw a YouTube video of the 2017 Eurovision Song Contest featuring singer Dami Im swiping holograms “Iron Man” style live on stage. I also saw footage of BASE Hologram’s Maria Callas and Roy Orbison holographic shows with accompanying orchestras. Ding ding ding!  It turns out that a newer technology had emerged where a projector could cast 2D digital images onto a special nearly transparent scrim that gave the illusion of 3D holograms. A bit of sleuthing around led me to Hologauze, Showtex, and eventually Kaleida and their scrim Holonet (known for Kanye West’s holographic “resurrection” of Kim Kardashian’s late father), which was much simpler to set up, use, and transport, plus budget-wise was within reach.

If the hologram cellist is not a real AI machine, is there any actual Artificial Intelligence involved in the production?

Adam: — So a colleague of mine in the Computer Science department [at Occidental College], Justin Lee, we were both hired the same year. I started talking to him about this piece and he immediately said that there are AI out there that not only can replicate your music but they can actually digest and learn your music, and then start to compose something new. That led to further discussions and then I primarily started working with Kathryn Leonard, chair of the department.  She and her partner Ghassan Sarkis, who teaches at Pomona College, together they built an AI for us named AGNES, for Automatic Generator Network for Excellent Songs.

So, it then became a collaboration between the three of us and the idea was to send representative works of mine that were robust so that we could feed this AI a lot of information. We fed it Picture Studies, which is my big orchestral piece, American Symphony (big orchestral piece), Losing Earth, which is a massive percussion concerto, and then my solo cello piece that Yves played many years back, which is called Ayudame. So three big works, so it could learn my whole language, my sense of color, and then a singular cello piece, so it could understand that this was the main element that we wanted it to take away.

After four solid months of training the AI, Kathryn and Ghassan asked me to send samples of material [from Automation] for it to spit out some music. They asked me if I liked how it was learning, and at the time it was learning in a very basic manner, like everything was quarter notes and eighth notes. I could see how it was learning, but it wasn’t growing. It’s very square and as if it’s trapped within a box. How can it really learn more?  They gave it more parameters, fed it more information and then in December when I had a full draft of the piece done for Yves, we fed it the whole section dubbed “Learning Mode” and later “Battle Mode”. After a week of learning that, they sent me back what it put out. And from the beginning, I said I would stay true to whatever the AI wrote and that I’d use it without altering it. So I said, “Alright,” and just embedded it into the score, sort of without looking at it, and that became the AI part. I wasn’t going to alter that process.

What is a halldorophone?

Adam: — The moment we discussed artificial intelligence and the idea of man vs machine, I went into the deep end and started looking for instruments and possibilities. That’s when I discovered NIME (New Interfaces for Musical Expression), that bi-annual conference where people are making these self-made instruments that have their own technology. That’s when I discovered BELA, this AI-operative computer box that can be attached to instruments, which led to me finding out about the halldorophone and [Oscar-winning composer] Hildur [Guðnadóttir] and the scores to Joker and Arrival.

It led me to Halldór Úlfarsson, who built the halldorophone, this cello-like electro-acoustic feedback instrument. In many ways, the halldorophone also represents AI. You try to control it by playing it like an acoustic instrument, but it is aleatoric – you can’t control what comes out.

Yves: — The halldorophone looks kind of like a cello. But it doesn’t function or sound anything like it. I’d describe it as a drone-based instrument with lots of distortion, vibrating metallic parts, frequency beats, rich overtones. One is constantly searching for new sounds and you never know how the instrument is going to react. Somehow, its weird, mesmerizing sounds just draw you in.

It is definitely a vital part to Automation. When Adam came to me with the idea of featuring the halldorophone in the work, I had my doubts. It was in the middle of the pandemic lockdown. The premiere was going to be delayed one if not two years. I had just had my baby boy and was sleep deprived. All I could think of were practical issues like how am I going to travel with this instrument, my cello, all the electronic and visual gear, to every future concert?  But Adam dragged me back into our usual “Dream Big” mode and convinced me how it fit perfectly both conceptually and sonically. He asked Occidental College to commission a new instrument, and we’ll leave the saga of how we got the instrument into the US from Europe during COVID-19 flight lockdown for another day. For me, it was the haunting score to HBO’s Chernobyl that sold me. Eerie, creeking, humming sounds. The feeling of lost souls.

I see the halldorophone from the human side of the same coin as Adam. After an epic “Battle Mode” where the human and AI cellists trade blow for blow, orchestra, electronics, visuals, lights, just about everything cuts out leaving this distraught, groundless moment – the human is alone with the halldorophone, vulnerable on stage. He questions his worth, of being replaced in the face of a more efficient, superior AI machine. He sees himself being automated. Only when the orchestra sneaks back in underneath the drone of the halldorophone with a breathtaking chorale is he reminded of the soul, of beauty, of imperfection, of life – all the things that make us human. It is a deeply spiritual moment that I hope I can share with the audience.

Halldorophones, AI, orchestras, holograms? This sounds like a massive production. How were you able to produce it?

Adam: — I have to start off by saying we have the definition of a 21st-century artist here in Yves. Someone who is ambitious, who has an idea, wants to make it happen and is resourceful, raises the money, finds the composer, and together with the composer’s connections, they find a way to put it on. How awesome is that?

Yves: — Aww, thanks, Adam!  I mean, this might quickly deteriorate into a bro-mance moment, but I’d start by saying that this is the sort of project, on such a scale, with so many moving parts, that I could not have done with any other composer other than Adam. He’s a dear friend of mine and we go way back to our doctoral days at Juilliard. And aside from being one of the best composers I know, with such a blend of film and concert influences, who respects tradition but is in touch with current trends, knows how to connect with an audience, and whose works are being played all over the US and abroad, he’s also a fellow “Dream Big” kind of artist that I could feed off of and feel nourished.

It’s already a big undertaking for an artist to produce a work like this solely from the creative standpoint. But to also take on the entrepreneurial aspects – fundraising, administrative, marketing – Automation truly felt like a mountain to climb at times. I knew that given the current slashed budgets for orchestras, I needed to find a way to make it impossible for them to say no. That started with raising the funds myself for a project that easily triples the budget of a usual concerto commission. With my entrepreneurial experience, particularly with nonprofits I have directed, I got to work writing grants and tapping into my network of supporters. I happened to be having an impromptu bite with my good friend Justin Sullivan, businessman, philanthropist and music lover, when he asked me if I had a dream project. It was in the early stages, but I told him about Automation. Ever the one to help people fulfill their potential, Justin also wanted to find a way to honor his late son Alec. It just all felt right. And so he commissioned the work.

We still needed the right orchestra and certainly the right conductor to help bring this to life. Someone who saw our vision, who was accomplished and respected, who was probably young and very much with finger on the contemporary pulse – it had to be Teddy Abrams, the Louisville Orchestra and their rich tradition of birthing these new works.

Through colleagues, I met Mauricio Ceppi (Funktaxi_1533), visual artist and celebrated VJ to all the NYC downtown nightclubs, who has been an invaluable visual consultant, along with his wizard VFX/digital artist Ryan Wise (DASYSTEM) whose technical and creative input have literally brought to life our AI cellist and more.

And, lastly, before the music starts playing me off the stage, I wouldn’t have been able to do it without the love of my life Renae, who keeps pushing me to go for it despite the obstacles, and my little boy Sebastien who looks up at me with wonder in his eyes, nor all the endless support over the years from my parents, teachers, and friends. Believe it or not, that is the short answer!