18 APRIL 22, 2011 | THE BOSTON PHOENIX | THEPHOENIX.COM

SCOTT M. L ACEY

fMusic is data. A shitload of it packed

in every single song. To people, music

equals entertainment. To a computer, it’s a

precise stream of ones and zeros.

If you could teach computers to under-

stand those ones and zeros — to digest them

like we do — then you could manipulate

sounds in ways even Prefuse 73 hasn’t

dreamed of. You could use your iPhone as

a violin. You could settle scores over who

liked an artist first — you, or your obnoxious

friend with the impossibly skinny jeans. You

could take your favorite Jay-Z album, tweak

one cut into a two-step swing remix, add

cowbell to another, and determine which

bangers the Grateful Dead would have been

most likely to cover in the early 1970s — all

on your smart phone, while in line for a

Metallica show.

You can do all this now. And you can

thank the Echo Nest: a small Somerville

company founded by two audiophilic tech

heroes, who have given developers every-

where the tools to shape the future of music.

The Echo Nest is so far ahead of the rest

of the music industry that their 15-minute

demo at the South by Southwest tech sum-

mit last month looked like a magic trick —

even to some of the highest-paid engineers

from the world’s biggest multimedia behe-

moths. After the Echo Nest’s presentation,

attendees rushed the microphones — all ask-

ing some form of the question, How the fuck did you do that?

The short answer: the Echo Nest has

analyzed tens of millions of songs, as well

as virtually every fan and artist profile, blog

post, comment, and article about music

that’s available online. Through that pro-

cess, they’ve amassed an incredibly dynamic

war chest of data — their Application

Programming Interface (API) — that can

power music apps in radical new ways. With

these extensive resources in their clutches,

the Echo Nest is the platform on which the

next generation of music fans will discover,

follow, remix, and recommend their favorite

bands. Most of what the company’s intelli-

gence can do hasn’t even been invented yet.

CEO Jim Lucchese says the Echo Nest

doesn’t want to be the next MTV, or the next

Spotify, or the next Island Def Jam (IDJ).

Instead, they’ve sealed watershed deals

with those companies and many more, all

of which are tapping the Echo Nest API to

gauge what’s hot and what’s not for who,

what, when, and where. In its search of

the next killer app, IDJ has even given Echo

Nest’s community of 7000 independent

developers access to more than one million

songs to play with.

Yet despite receiving tons of tech press,

the Davis Square company remains in the

shadows of pop culture. They’re not even as

well-known as Pandora, which, according

to Lucchese, has taken a decade to index the

same number of songs that his crew could

crunch in two hours. For their scrapper sta-

tus, Lucchese credits the fact that Echo Nest

provides no direct-to-consumer products,

and concedes that the business is hardly run

by marketing mavens. But he suspects they

won’t be underdogs for much longer.

“If there’s one thing that we’ve done a

terrible job of, it’s driving more mainstream

awareness to the awesome shit that people

are doing on our platform,” says Lucchese, an

attorney. “Maybe it’s because we’re founded

by two MIT PhDs and run by a lawyer. In

the developer community, we’re past being

considered the ‘smart little guys,’ and we’re

being recognized as a best-in-class platform.

Granted that it’s still kind of the early days

for [everybody else] to really understand what

we do, but now it looks like mobile apps are

changing that.”

AGREE TO DISAGREEIn the late 1990s, Brian Whitman was a frus-

trated computer scientist and electronic mu-

sician living in New York City, and perform-

ing under the name Blitter. He played regular

gigs and even dropped some vinyl, but spent

most of his time imagining ways to get his

music in front of potential fans. At the time,

he noticed that online message boards were

becoming increasingly populated by people

who were anxious to discuss — and, more

often, argue about — music trends and styl-

ings. Somehow, Whitman thought, these

were not disparate musings.

“In hindsight,” he says, “this is all very

obvious. But when these communities were

forming, the people who were doing music

recommendation and retrieval weren’t look-

ing at [blogs and message boards] — they

were just looking at the audio signals. . . .

I set out to prove that the more you know

about a community, the more you under-

stand peoples’ preferences.”

Following his interest in the conversa-

tion around music, Whitman’s explora-

tions brought him to the world-renowned

MIT Media Lab, where he eventually met

his philosophical nemesis and future busi-

ness partner, Tristan Jehan: a soft-spoken,

French-born computer scientist, amateur

keyboardist, and researcher who cut his

teeth at UC-Berkeley’s Center for New Music

and Audio Technologies.

NEST BUILDERS From left, Tristan Jehan, Jim Lucchese, and Brian Whitman in the Echo Nest’s Davis Square headquarters.

THE MOST IMPORTANT MUSIC COMPANY ON EARTHTHE ECHO NEST IS READY TO FLY_BY CHRIS FAR AONE

THEPHOENIX.COM | THE BOSTON PHOENIX | APRIL 22, 2011 19

Jehan’s view of music analysis was the

opposite of Whitman’s: he thought that

relationships between songs should be

derived by extracting and analyzing musical

metadata. Jehan came to MIT to prove that

sounds — as opposed to the worldwide dia-

logue about music — were the best barom-

eters of listener taste.

“I’d been working on how to make com-

puters better understand music,” says Jehan.

“Brian was looking at how computers could

understand music in the context of how

people speak about it on the Web.” Adds

Whitman: “You can’t just look at the audio

signals, and at the same time you can’t

ignore the audio — you have to know

what the song sounds like, and under-

stand the conversation around it. You

need to do both.”

By the time that Whitman and

Jehan earned their doctorate degrees (in

machine listening and media arts and

sciences, respectively) in 2005, both were

considered all-stars in their parallel fields.

So it was fitting that they agreed to disagree,

and partnered to launch the Echo Nest out of

a small office in the same building where the

company now occupies several suites.

Using their complementary research as a

foundation, they wrote programs that crawl

the Internet (and streaming services like last.

fm), analyzing everything from comments

and discussion about songs and artists, to

the rhythm, harmony, and timbre of mil-

lions of actual tracks. Within two years,

they’d built the most powerful interactive

music database ever indexed as a single plat-

form — an API they would come to call the

“Musical Brain.”

But while Whitman and Jehan had ideas

that could significantly impact listening

habits the world over, they still hadn’t fig-

ured out how to use their Musical Brain to

make money. “It was a pretty classic example

of two scientists running a business,” says

Whitman.

Luckily for them, the right people noticed

Echo Nest relatively early on. They attracted

investors like Barry Vercoe, a music-program-

ming icon and a co-founder of the Media

Lab, and Don Rose, who co-founded the New

England–based imprint Rykodisc.

“They didn’t have any paying clients, and

they didn’t have any products, but I imme-

diately liked them,” says Rose. “Before I did

anything with them, though, I wanted to

see if they could make it work. So I had them

sniff my iPod to make recommendations,

and lo and behold, what Pandora did manu-

ally over time, they were able to do in less

than a day.”

Rose’s son, Scotty, who now works at the

Echo Nest, was in a band with Lucchese. After

Lucchese learned about the company, he left

New York to join the Somerville operation full-

time and help write a business plan. He was

familiar with the music-licensing gauntlet

from working with platinum artists at the

mega-firm Greenberg Traurig, and had the

perfect tough-but-friendly attitude to navigate

the industry for them.

With Lucchese on board, in 2007 the

company was awarded more than $500,000

in grant money by the National Science

Foundation, and that year also attracted

the attention of the Boston-based Kelso

Management Company investment fund.

From there, the Echo Nest launched its devel-

oper API at the DEMO conference in Palm

Desert, California, in September 2008.

At the time, it was becoming ever more

clear that music search-and-discovery

tech was of paramount importance to the

industry. The leader in that arena was

Pandora, which had just launched a mobile

version of its software.

People tapped Pandora to direct them

toward music based on the songs they

already liked. But Pandora’s patented Music

Genome Project employs actual humans to

manually index tracks according to audio

criteria. Decisions as to what songs fit which

parameters are subjective, made by a person.

The Echo Nest’s approach, alternatively,

couldn’t be any more different. It is entirely

algorithmic. The Musical Brain makes those

judgment calls better than any teams of

humans could, Jehan and Whitman say

— and much, much faster.

Still, despite the Echo Nest’s booming

popularity in tech circles, it would take a

calculated sales pitch to entice big labels and

media companies.

“Everybody had already seen a lot

of PhDs give them the bullshit line

about having the science to understand

music,” says Lucchese, “so we had to

separate ourselves from the scores of ill-

conceived companies that were doomed

to fail from the jump.”

API IS THE NEW MP3On a now-storied fall weekend in late 2009,

about 200 techies gathered at Microsoft’s

New England Research and Development

Center on Memorial Drive in East Cam-

bridge for the first stateside Music Hack

Day, which, according to Lucchese, is

“essentially a jam session” for music-tech

visionaries. For two straight days — some

participants stayed awake on Red Bull and

coffee the whole time — hackers, program-

mers, and developers tapped the Echo Nest

API (and those of other participating com-

panies) in order to build apps that change

the way people find and digest music.

“You used to have all of these gatekeep-

ers,” says Paul Lamere, director of the Echo

Nest’s developer community. “There were

the A&R folks, the producers, the labels,

the old-time DJs, the record store buyers —

they were all the tastemakers. Now that’s

all gone, which means people have to be

able to search through everything on their

own and make these decisions — and that’s

where the Echo Nest comes in . . . [Music

Hack Day] is a business-free zone, but it’s

great because kernels of ideas from there

have found their ways into real products

that help fans find what they want.”

Lamere sits at the helm of Boston’s

annual hack day, and of others in Cannes,

Stockholm, Barcelona, Amsterdam, Berlin,

London, and San Francisco. A guru’s

guru in music innovation, Lamere came

to the Echo Nest in early 2009 from Sun

Microsystems, where he’d spent the previ-

ous five years researching music explora-

tion. Though excited about his work there,

Lamere, whose kids describe his music

taste as “dad-core,” says he became frustrat-

ed that Sun never intended to introduce his

inventions to the marketplace, and defect-

ed to the Echo Nest as a result. Within

three months of his arrival, the company

inked a deal with the music streaming

service Spotify to assist with playlist and

discovery services.

At hack days and on a daily basis, Lamere

helps guide inventors toward apps that are

already revolutionizing the commercial space.

Last December, the Echo Nest teamed with

Viacom for the MTV Music Meter, which pegs

the hottest rising artists on the Web, then

helps pair those acts with mtv.com readers

who might dig them. That same month,

they also announced a similar venture with

the British Broadcasting Company that helps

visitors to the BBC’s Music Showcase hub find

favorable clips and videos.

“We don’t care where the songs come

from,” says Lucchese. “What we do is sit on

top of the data and enable the experiences

through which people access [music]. We’ve

built our whole business on the idea that,

one day, kickass independent developers

will be able to build apps and release them

legally without ever talking to a lawyer.”

The Echo Nest’s biggest landmark deal

so far came this February, when they linked

with IDJ to bring developers closer to license

holders. With Lamere assisting, app wizards

can now write programs using the Echo Nest

platform — for everything from PCs to smart

phones — and release them commercially

with no hassle. Considering the depth of

IDJ’s roster, which includes everyone from

Ghostface to Fall Out Boy, the relationship

between the Echo Nest and IDJ has the poten-

tial to yield countless retail opportunities.

“My company is always looking to inno-

vate,” says IDJ senior vice-president of digi-

tal and business development Jon Vanhala,

who helped shepherd the deal through.

“There’s no silver bullet in the app world

right now — nobody can tell you what’s

absolutely going to be cool or not. You have

to have some faith, and then let the market

decide. . . . We believe this community of

developers is exceptionally creative, and [the

Echo Nest] gives us access to all of that.”

SOMETHING FOR EVERYONEIf hack days are the Seattle grunge scene

2.0, then Echo Nest is Sub Pop circa 1991. At

least that’s how it seems now that the larger

music world is recognizing how integral app

advancements are to steering the industry

forward. On April 28, MTV will host its first

annual O Music Awards (“to celebrate online

creativity”), where Lamere and his teenage

daughter Jennie are up for Best Music Hack.

Their nominated app, Jennie’s Ultimate

Road Trip, uses the Songkick and Echo Nest

APIs to help music fans find tour dates for

their favorite artists in each town along any

travel route.

The Lameres engineered their app at

last year’s Boston Music Hack Day, as

did two of the other OMA nominees for

Best Music Hack. In fact, these powwows

have become ground zero for cataclysmic

developments, from the aforementioned

Invisible Instruments iPhone violin app,

to sQRatchLive, which allows users to

instantaneously buy whatever track a DJ is

playing in the club.

Moving forward, the Echo Nest’s new

visual remix tech will allow users to person-

alize music videos in real time, so that your

friends’ ugly faces — rather than Kanye’s

— will fill your screen. Equally awesome are

their soon-arriving “query-by-description”

apps, which are essentially digital versions

of the grease-ball snots behind record-store

counters to whom customers hum songs.

To keep tabs on all this, the Echo Nest even

launched its own blog, evolver.fm, to serve

as a critical clearinghouse for breaking

music-app developments.

“There is something for everyone out

there — it’s just a matter of finding what

works for you,” says evolver.fm editor Eliot

Van Buskirk. A former writer for CNET and

Wired, Van Buskirk has surveyed the digital

landscape for more than a decade; Echo

Nest hired him to review music apps and

evaluate the marketplace. “I’m testing an

app right now that quizzes you about what

you’re doing this weekend, and then picks

songs to go with that. And that’s kind of the

point to a lot of this — it’s fun, it’s free, and

people who aren’t music geeks are now able

to do some incredible stuff.”

“We understand the entire language

of music better than anyone else to date,”

says Lucchese, “and now we’ve put that

research and analysis into the hands appli-

cation developers. . . . To us, there’s no

doubt about it — this is the new class of

creative people who are reshaping the role of

music in our lives. With them, we’re aim-

ing toward where things are heading, and

making sure that we’re still around when it

explodes.” ^

Chris Faraone can be reached at cfaraone@phx.com.

‘It’s fun, it’s free, and people who aren’t music geeks are now able to do some incredible stuff.’

RETHINKINGMUSIC