AI Blues

What happened when I tried to replace my favorite DJ with artificial intelligence.

Still, it’s hard to imagine AI making a real emotional connection, touching us as the best media personalities do. Could a robot ever be a great stand-up comedian? Will your favorite TV talk show one day be hosted by an algorithm? Ridiculous, right? Perhaps. But just a few decades ago, it was ridiculous to imagine we’d soon be carrying the equivalent of a super-computer in our pockets.

I’ve been thinking about these questions lately because of my musical tastes. I love jazz, a genre that isn’t wildly popular right now. My kind of music hasn’t topped the charts for ages, but there’s a mountain of recordings going back nearly a century to be mined and enjoyed. This trove is way too large for me to efficiently scour on my own, but it’s a perfect assignment for the AI-driven recommendation systems used by digital music services like Spotify and Pandora.

A few years ago, I decided to give it a try. Pandora  was appealing because its musical storehouse is not only very large — more than 2 million songs — but extensively annotated by human experts who have been at it for years. If I’ve learned anything in my research at the MIT Media Lab, it’s that the machines do much better with human oversight and guidance. So I created a Pandora account, found a pre-existing station dedicated to the jazz subgenre I like best, the Hard Bop of the late 1950s and early ’60s, and … almost immediately lost interest.

The reason: The Pandora station couldn’t compete with an analog-era radio show to which I’d been devoted for years, “Jazz with Bob Parlocha.” I discovered it back in 2005, while driving down a lonely Cape Cod highway one morning about an hour before sunrise. At one end of the FM dial, I came on this low, gravelly voice discussing several cuts that had just played. There was something about the host’s chill delivery — confiding, patient, off hand yet intimate — that drew me in. And for the next 90 minutes, I was amazed at how closely his taste aligned with mine.

I started tuning in whenever and wherever I could. On most of the public-radio stations that carried the show in syndication, it was a late-night or overnight fixture. Due to time-zone differences, there was a long stretch of each day in which Parlocha (“par-LOH-chuh”) could always be heard somewhere. Many nights, I fell asleep to one airing and woke up with the alarm to another. When traveling, I’d use my phone and the hotel Wi-Fi to get a dose before starting the day.

Last year, Parlocha died suddenly of a heart attack. Although I knew next to nothing about his life, an obituary I found online revealed that he was 76, a former psychiatric nurse at the University of California at San Francisco Medical Center who played saxophone and flute on the side. In the late 1970s he’d submitted a tape to a Bay Area jazz station looking for new DJs and was hired. By the time he died, his show was a fixture on dozens of public radio stations around the country.

For months, I got by on repeats some stations used to temporarily ill the time slot, but those eventually trailed off. Then it hit me: Why not use technology to create a Parlocha replacement? Artificial intelligence (AI) plays a role in my work at MIT’s Laboratory for Social Machines. I’ve seen it in action and believe in its potential to help society meet all kinds of challenges. For instance, my research group has a project called the Electome in which we’re using AI to track and analyze the social-media response to the current presidential election.

By comparison, replacing a departed jazz DJ is a small, idiosyncratic challenge. But in a way, it’s a metaphor for the big technology question of the moment: How close to human can machines get?

AI refers to computers programmed to simulate various kinds of human intelligence. It’s been around since the mid-20th century, but recently it’s been drawing a lot of attention, in part because of a wave of warnings from Elon Musk, Stephen Hawking, and others that these machines could become so intelligent, they’ll eventually take over the world.

Most scientists aren’t worried about evil robots appearing anytime soon. While the scare stories make great headlines, they can obscure the positive potential of AI. By focusing on the ways in which machines could defeat us — as they’ve already done in many games, including chess, Jeopardy, and Go — we overlook that the original point of computers was to help us, work that has only just begun.

For centuries, media companies have been brilliant at using new technologies to deliver stories and other content people love. In the digital age, Amazon was a pioneer in the use of AI-based recommendation systems that employ customer purchase histories (our own and those of people with similar tastes) to suggest books and other items that the algorithms predict we’ll like — a methodology called “collaborative filtering.” The same type of AI is behind the personalized headline lists offered to digital New York Times subscribers, and the “suggestions for you” filter on Netlix. I knew that AI drove the music recommendations at Pandora, where my old account lay dormant.

Could it take the place of a sorely missed DJ?


Robot Parlocha

Anyone who’s used Siri knows how far AI is from reasoning and speaking like a person, let alone taking on a subject as complex and nuanced as musical taste. I’d watched the hilarious IBM ad in which Bob Dylan chats with Watson, an AI system that thinks it has singing talent and shares a painful sample.

But I also knew that AI researchers have been on a tear, thanks to the growing ability of computers to learn from large bodies of data. It’s called machine learning, and in simple terms it works like this: A computer system is fed amounts data far beyond the processing power of human intelligence (Watson brags to Dylan that it can read 800 million pages of song lyrics per second). Combing through it all, algorithms uncover new patterns and connections that the machine uses to make future decisions, an artificial version of the brain’s associative powers. This is the frontier of computer science. Some of its more impressive feats, such as Facebook’s automatic photo recognition and tagging, have become so familiar that we already take them for granted.

My hope was that AI could simulate the very particular musical taste that drove Parlocha’s work. In theory, a system could be programmed to play the jazz artists and styles that defined his playlist. And with the help of machine learning, it might replicate another key aspect of the late DJ’s toolbox: surprise. For example, when I started listening to Parlocha, I already knew the more prominent guitarists he played, including Grant Green, Joe Pass, and Jim Hall. But he often brought in terrific lesser-known artists I’d never have found on my own. Theoretically, if I inputted all the guitarists I already liked into an intelligent system with access to a large jazz database, it could do the same thing.

So I went back to my old Pandora account with a new mission: use the company’s Music Genome Project, the vast database it describes as “a deeply detailed hand-built musical taxonomy,” to a create a machine-driven simulacrum of the Parlocha guitar aesthetic.

The Pandora system has two basic elements. First, it exploits the information you give it about your own tastes, playing artists and songs it knows with high confidence you’ll like based on the information you provide when you create a station plus the ratings you give songs as the station evolves. Then, it explores broader possibilities beyond your cues. It does so using links it has already established between artists, instruments, songs, styles, and other factors such as the timbre, color, and quality of the sound, along with cuts that users with tastes similar to yours have enjoyed. Most consumer recommendation systems, from restaurant apps to online clothing stores, use some version of this two-step approach.

Inspired by Parlocha’s taste, I sat down and created a few jazz stations, one of them dedicated to guitar. I chose Wes Montgomery, the great mid-20th-century guitarist, as the starting point or “seed” for the station. The rest seemed simple enough: Based on that, Pandora would proceed to play Montgomery-ish cuts from all sorts of guitarists, while I would use the system’s curation options — you can give any song a thumbs-up or thumbs-down rating — to guide it toward the exact sound I was seeking.

I was a diligent curator, listening often and carefully “thumbing” songs up or down. I wanted pieces that were either solo guitar or guitar-led, and with a certain relaxed, introspective feel for which Parlocha had a penchant. I up-voted songs that had a lazy or mid-tempo swing groove and, in some cases, “cool jazz” influences. I didn’t know if Pandora’s AI paid attention to station titles, but as a further clue, I named mine Quiet Jazz Guitar.

I listened to the station while working. I listened in the kitchen while making breakfast and dinner. And I frequently gave songs the thumbs-up or down. Yet try as I might, it didn’t approach my ideal. No matter how often I rejected noisy, up-tempo pieces that didn’t meet my deinition of “quiet,” it plied me with more. I wanted cuts with the guitar in the lead, but it consistently force-fed me numbers in which that instrument was buried. Months into the experiment, it was still frequently playing pieces with no guitar whatsoever. Technology that was supposed to make it easy to design my dream radio station was making it harder.

So I got in touch with Pandora and talked to Erik Schmidt, a senior scientist at the company whose expertise is music-information retrieval. I allowed him to look at all my data. “Your example that you presented is particularly challenging,” he said, with the delicate tone of a doctor telling me I had something terminal. I’d been a very good curator, he acknowledged, notching 160 up/down song ratings since I’d started the station, and that was great. “The thumbs you gave us, this is a huge help for us.”

But I’d misunderstood something basic about how the Pandora system works. By seeding my station with a particular artist, I’d unknowingly instructed the machine to offer other artists similar to Wes Montgomery based on a huge array of factors. Instrumentation was one, but the wisdom-of-the-crowd that helps drive the “explore” function might not consider the presence of a guitar an absolute prerequisite for my station. Songs with similar emotional qualities, or by pianists or drummers with whom Montgomery played, might be selected. With thumbs alone, Schmidt said, it was difficult for the machine to focus the station to the point where it played only jazz guitar.

And I’d made another mistake. In my frustration, I had tried to help the system by using Pandora’s “Add Variety” button to tell it explicitly about 11 other guitarists I liked. This had sent it off on a sprawling journey across scores of musicians and musical works to which these artists are tied, spinning cuts by pianist Oscar Peterson, saxophonist Hank Mobley, and many other non-guitarists.

Schmidt told me I should have seeded the station not with a particular artist but a single song that exemplified what I was seeking. That would have instructed the AI to look only for similar songs, period — no broad sampling of works by related artists.

Why didn’t the machine recognize that problem? Weren’t my many thumbs-downs a clear signal that I wasn’t satisfied? Couldn’t it have put a virtual hand on my shoulder and said, “You seem unhappy, William. Is something wrong?” It turns out that Pandora’s limited curation options are an intentional part of the system, which aims to keep the feedback “simple and direct,” as Schmidt put it, so the algorithms do the heavy lifting — which, after all, is the point of AI: make life easier for the humans.


The Automatic Ideal

Schmidt spoke with excitement and pride about Pandora’s ongoing effort to solve “the never-ending recommendation problem” in a way that doesn’t impose huge demands on users. It’s an admirable goal, but it has costs. In this case, the machine wasn’t serving my needs. I was willing and able to give it more input, but it was effectively designed not to comply. Rather than becoming my new jazz mentor and virtual friend, the technology had left me exasperated.

The problem goes back to an old fault line in computer science. Among the field’s pioneers, there was one school of thought which held that AI was on a path to exceeding human intelligence, and the point was to get out of the way and let the machines do their stuff. This produced an ideal of pure automaticity — the better the machine, the less human input it needs — that remains influential today.

There are countless ways in which automaticity serves us well by relieving us of drudge work. I love how my iPhone quietly scans my emails to suggest additional email addresses and other details I can add to my existing contacts.

But there are also drawbacks. Though Parlocha’s show was a human creation, in one sense it was a completely “automatic” experience: it asked nothing of me but passive enjoyment. I could have emailed him feedback; he often gave out his address, promising he read every message. But I never felt that was necessary. The show was fantastic exactly as it was, because it was the product of his unique experiences, personality, and sensibility. Machines don’t have any of those things. They’re not people, and no matter how intelligently they’ve been coded, never will be. In order to meet our needs, they require meaningful feedback from us.

This points to where the potential of AI really lies. For a computer-driven music service to produce the kind of satisfaction I got from the relationship I had with my favorite DJ, it has to be a collaboration with someone on the other side of the screen.

In some corners of the media business, user input has a bad name. This is partly about the enduring attachment to automaticity, which, among other things, can yield huge cost savings. And it’s partly the result of failed experiments. In the news business, the disastrous track record of reader comments — due mainly to cranks and trolls — is a cautionary tale: Beware of people, they can be so horrible. True enough — but so can computers if they don’t know what you really want.

On the other side of the philosophical fault line, there’s a school of thought that says machines and humans should work together to solve problems. Its patron saint is J. C. R. Licklider, a mid-20th-century computer scientist who believed it was important for people and machines to have a partnership “a very close coupling” analogous to that of colleagues collaborating on a project.

Licklider believed it made perfect sense for computers and people to work together because we have different strengths. He used an analogy from nature, about a species of wasp that lives symbiotically inside a fig tree: “The tree cannot reproduce without the insect; the insect cannot eat without the tree; together, they constitute not only a viable but a productive and thriving partnership.”

Licklider predicted that it wouldn’t take us long to develop such a symbiosis with machines. Once it existed, we could use it productively for as long as it took computers to catch up with human intelligence. He conceded it was hard to put a number on how long that would take. It could be 15 years, or 500, “but those years should be intellectually the most creative and exciting in the history of mankind.”

He had this lovely vision in 1960, when Bob Parlocha was 22 years old. In a way, human-computer symbiosis has been slowly emerging ever since. But would anyone argue that we have achieved the “productive and thriving partnership” that Licklider imagined?

As for me, I took Schmidt’s advice and created a new jazz-guitar station based on a single song. It worked out better than my previous effort; but in the meantime I stumbled on a college radio station with an excellent jazz show. Depending on the host, it can come uncannily close to the Parlochian ideal.

One morning not long ago, I called the station to let them know. The DJ answered the phone himself, obviously thrilled to be hearing from a live person. He thanked me effusively.

Maybe one day I’ll have the same mutually grateful exchange with a machine.

We will be publishing our readers’ best feedback and incisive dissent; send your thoughts to