A sound equation: UWM mathematician works at the intersection of music and technology

Georg Essl

Think about a rubber band and the many ways it can be twisted or contorted. Now, think about stretching it and plucking it with a finger, and the slight “boing” noise the action makes.

This is the creation of sound, which is the basis of Georg Essl’s work. But exactly how the UWM mathematical sciences research professor creates sound might stretch the imagination.

Essl’s work can involve complex algorithms and monitors displaying oscillating waves that look like they’re straight out of an old sci-fi TV series. It can also involve musical instruments, everything from electric guitars and keyboards to a simple Tibetan singing bowl, a bell-like instrument played with a mallet to produce a deep, reverberating sound.

He works in sound synthesis, a field within music technology at the intersection of music-making and computer science, math and engineering. The outcome is sound that has many possible applications, from being incorporated into a piece of music to serving as a sound effect for movies or video games. A recording artist might want to sample a synthesized sound for new music. A big-budget sci-fi movie director might need a new “whoosh” for a flying spaceship.

“I see my role more as an instrument builder,” Essl says. “I’m the one who builds the capability that others can use. I would rather have those ideas spread out and flourish.”

Essl earned a 2019 Guggenheim Fellowship in computer science . The Guggenheim is a prestigious award that recognizes prior achievement and exceptional promise of scholars, artists and writers. He was one of 168 fellows chosen from a field of nearly 3,000 applicants.

The Guggenheim project focuses on using topology, which is related to geometry, to create algorithms for producing sounds, such as those heard when using electronic keyboards or synthesizers.

Essl’s work has broad reach in popular culture, especially in the increasingly lucrative field of online video gaming with its countless virtual worlds. The Entertainment Software Association says the United States video game industry generated a record $43.4 billion in revenue in 2018, up 18 percent from the previous year.

The field seems ideal for a man whose father was a physicist and whose mother made sure the family was exposed to music education. As a child, Essl would play on early home computers provided by his father. Essl also learned to play the recorder and piano, and he became interested in music composition.

Today, his work is largely centered around crafting new instruments and making new sounds that can be used in interactive music compositions or other audio applications. It’s resulted in some intriguing, if unorthodox, projects.

For example, before he arrived at UWM’s College of Letters & Science in 2016, Essl taught at the University of Michigan, where he founded and directed the Michigan Mobile Phone Ensemble. Yes, a nontraditional symphony of cell phones creating sounds via information received over each device’s sensors, including the touch screen, GPS and microphone.

His work keeps him on the lookout for new ways to produce new sounds, and even a rubber band can be a source of inspiration. A tiny variation in the distance or shape of a stretched rubber band will alter the sound, however slight, and the combinations resulting from twisting and turning the material might be endless.

“So imagine sound synthesis behaves like deforming rubber bands,” Essl says.
He can research the possibilities further by plugging the rubber band’s dimensions into a computer program. He writes his own software, which allows him to explore new sounds without worrying about the rubber band’s physical constraints. What kind of sounds are possible if it could be stretched with no concerns about it breaking?

And instead of spending hours contorting the rubber band into different shapes, the task can be accomplished with just a couple of keystrokes.

Another type of sound synthesis involves using mathematical equations to devise ways of creating sounds that otherwise may not physically exist. For example, what kind of sound would an object with the dimensions of a Tibetan singing bowl make if the bowl was dented in one spot versus another? What if that bowl was wavy instead of cylindrical in shape?

“We can then go from that object and start doing things that maybe are not so physical, that we can’t easily do in the physical world, and modify the sound,” Essl says.

Not all of the sounds he creates require complex computer algorithms. He is, after all, a classically trained pianist who dabbles in string instruments, so he can harness his research to create his own music.

But he’s mainly interested in the process of creating the sounds themselves instead of sewing them together into a work of art. Essl is happy to have others incorporate his work.

“I’m going to give you a new way to create a lot of noise,” Essl says. “We’re going to do a lot of math to do that.”

By Genaro C. Armas, University Relations