How Streaming Services Recommend Music

Would you rather a human or algorithm recommend an album to you? Oddly, the processes are sort of similar.

Last Friday, I appeared on a podcast produced by The Economist called The Intelligence where I discussed how introductions in popular songs are becoming shorter. This was super cool, but the reason I bring it up is because one of Spotify’s algorithms recommended that I listen to the episode. In many ways, the recommendation was right on the money. Who doesn’t want to listen to themselves? Regardless, it made me think it was a good time to dive into how these recommendation algorithms work.

How Humans and Machines Recommend Music

It’s Saturday afternoon. You decide to pop into your local record shop. A few days before, you listened to Bob Dylan’s album Bringing It All Back Home for the first time. You were infatuated. So you ask the local record clerk to recommend something similar but a bit more contemporary. She points you to Jason Isbell’s album The Nashville Sound. This makes sense. There is a good deal of overlap between Jason Isbell and Bob Dylan fans. Plus, both albums have a similar sonic palette.

In short, this record store clerk is considering what fans of Bringing It All Back Home also like, along with other albums that have sonic textures like Dylan’s masterpiece. What’s interesting is that the two most common kinds of algorithmic recommenders work similarly. They are known as content-based filtering and collaborative filtering.

• Content-Based Filtering: This methodology looks at the actual sonic characteristics, or metadata, associated with a recording – like BPM, loudness, genre, instrumentation, and release date, among others – to make a recommendation. This is the machine equivalent to recommending The Nashville Sound based on how it sounds similar to Bringing It All Back Home.

• Collaborative Filtering: This methodology looks at user listening habits rather than sonic characteristics. For example, if I listen to a ton of Bruce Springsteen and John Mellencamp, it’s going to find other users that listen to them and recommend me whatever else they are listening to. If those users are listening to Tom Petty, then it will serve me some of his music. This is the machine equivalent to recommending The Nashville Sound because fans of Bob Dylan often listen to Jason Isbell.

Most recommender systems you interact with are probably some combination of these methods. To see them in action, I decided to ask a few different systems to recommend 10 songs similar to “Ain’t No Mountain High Enough” by Marvin Gaye and Tammi Terrell. As a point of comparison, I had a friend of mine manually generate a list of recommendations.

Human
This list comes from my friend who is a music encyclopedia. As you’ll see, many of his recommendations don’t appear on any other list. It displays deep knowledge of the sonic qualities of “Ain’t No Mountain High Enough”.

1. “Ooh Carla, Ooh Otis” by Otis Redding & Carla Thomas

2. “Every Beat of My Heart” by The Du-Ettes

3. “(Your Love Keeps Lifting Me) Higher & Higher” by Jackie Wilson

4. “Just One Look” by Doris Troy

5. “We’re In Love” by The Impressions

6. “Baby, I Love You” by The Ronettes

7. “Be Young, Be Foolish, Be Happy” by The Tams

8. “Ain’t Too Proud to Beg” by The Temptations

9. “If You Love Her” by Bobby Taylor & The Vancouvers

10. “Someday We’ll Be Together” by Diana Ross & The Supremes

Audiomack
I not only work at Audiomack, but I specifically work on our recommender systems. So I’m going to be a bit biased here. Beyond the fact that this list is overly reliant on The Temptations, I think it is pretty good.

1. “Just My Imagination (Running Away With Me)” by The Temptations

2. “It’s a Man’s, Man’s, Man’s World” by James Brown

3. “Ain’t Too Proud to Beg” by The Temptations

4. “The Way You Do The Things You Do” by The Temptations

5. “My Girl” by The Temptations

6. “What’s Going On” by Marvin Gaye

7. “Try a Little Tenderness” by Otis Redding

8. “It’s a Shame” by The Spinners

9. “I Heard It Through the Grapevine” by Marvin Gaye

10. “Reach Out I’ll Be There” by Four Tops

---

---

ChatGPT
If you’ve read the news over the last few months, you’re probably familiar with ChatGPT. It’s a chatbot created by OpenAI. I prompted the bot to act as a song recommender. It did a pretty good job. The Jimmy Ruffin song was an unexpected but great selection.

1. “My Girl” by The Temptations

2. “Respect” by Otis Redding

3. “I Want You Back” by The Jackson 5

4. “I Heard It Through the Grapevine” by Marvin Gaye

5. “(Your Love Keeps Lifting Me) Higher & Higher” by Jackie Wilson

6. “Son of a Preacher Man” by Dusty Springfield

7. “Ain’t Too Proud to Beg” by The Temptations

8. “Try a Little Tenderness” by Otis Redding

9. “What Becomes of the Brokenhearted” by Jimmy Ruffin

10. “Soul Man” by Sam & Dave

Spotify - Person 1
I included two sets of recommendations from Spotify. The reason I did this is to demonstrate how Spotify uses both content-based and collaborative filtering, meaning that two people generating recommendations on the same song will get a different set of songs. This first list was generated on my account. I don’t think this list is very good. While I enjoy songs like “Beyond the Sea”, “Eight Days a Week”, “Boogie Shoes”, and “Everywhere”, they quite distinct in my mind compared to “Ain’t No Mountain High Enough”. “Got to Get You Into My Life” - maybe the most Motown of Beatles songs - is a great find, though.

1. “Got to Get You Into My Life” by The Beatles

2. “Dancing in the Moonlight” by King Harvest

3. “Do You Believe in Magic?” by The Lovin’ Spoonful

4. “Beyond the Sea” by Bobby Darin

5. “Ain’t Nothing Like the Real Thing” by Marvin Gaye & Tammi Terrell

6. “Everywhere” by Fleetwood Mac

7. “Eight Days a Week” by The Beatles

8. “This Old Heart of Mine (Is Weak For You)” by The Isley Brothers

9. “One Fine Day” by The Chiffons

10. “Boogie Shoes” by KC & the Sunshine Band

Spotify - Person 2
I think this version, generated on a friend’s Spotify account, is slightly better. But there are still some strange selections, like Bob Marley’s “Could You Be Loved”.

1. “Twist & Shout” by The Beatles

2. “It’s All Right” by The Impressions

3. “Surfin’ USA” by The Beach Boys

4. “Could You Be Loved” by Bob Marley & The Wailers

5. “Ain’t Nothing Like the Real Thing” by Marvin Gaye & Tammi Terrell

6. “Papa’s Got a Brand New Bag” by James Brown

7. “Eight Days a Week” by The Beatles

8. “Rich Girl” by Daryl Hall & John Oates

9. “Brandy (You’re a Fine Girl)” by Looking Glass

10. “Right Back Where We Started From” by Maxine Nightingale

SoundCloud
In my opinion, this is the worst set of recommendations. I’d only compare “Come On Eileen” to “Ain’t No Mountain High Enough” if I had a very limited set of songs to choose from. I will cut SoundCloud some slack. If this algorithm is largely powered collaborative filtering, this might be ideal based on what their users are listening to. Context aside, it is a miss.

1. “Stand by Me” by Ben E. King

2. “Play that Funky Music” by Wild Cherry

3. “The Rubberband Man” by The Spinners

4. “You Make My Dreams (Come True)” by Daryl Hall & John Oates

5. “Break My Stride” by Matthew Wilder

6. “Kung Fu Fighting” by Carl Douglas

7. “September” by Earth, Wind & Fire

8. “Superstition” by Stevie Wonder

9. “December, 1963 (Oh What a Night!)” by Frankie Valli & the Four Seasons

10. “Come On Eileen” by Dexys Midnight Runners

A New One
"Ice Cream Man." by RAYE
2023 - Contemporary R&B

Last week, I was doing some work at Audiomack around the gender composition of artist audiences. That’s when I came across the British singer-songwriter RAYE. Nearly 83% of her audience was composed of women. I was intrigued, so I clicked play on this track, a song from her debut LP My 21st Century Blues. I was awestruck by its lurching, understated confidence.

RAYE recounts being sexually assaulted by a producer on this track. While her account is harrowing, I celebrate her for coming forth with the story. The world would be better off if more men took RAYE and her story seriously.

An Old One
"Just Another Day…” by Queen Latifah
1993 - Conscious Hip-Hop

My girlfriend and I were watching the CBS drama The Equalizer a few days ago. It stars Queen Latifah. It’s odd that she is known more as an actor these days given that she was considered such a groundbreaking rapper in the early-1990s. On “Just Another Day…” you can hear Queen Latifah’s musical skills in full force. It makes you wonder why she hasn’t released an album since 2009.

---

Want to know why recent hit songs have 4 times the songwriters as older hits? Read my latest piece for Tedium.

Want to hear the music that I make? Check out my latest EP.