Day 16 of #100daysofnetworks

Spotify Crawling - Music Collaboration Networks!

I have a special gift for you all today! I created a simple crawler for building Music Collaboration Networks using Spotify. In order to use the Spotify crawler, you will need to have a Spotify account and get set up for authentication.

To make all of that much easier, I am using the Spotipy Python Library. You can learn about it here.

And you can use this to get setup on Spotify’s end.

Today, I used this to build a network of B.B. King’s music collaboration (or sampling). I got to see him twice in concert in the late 1990s, and he was just amazing.

But First…

Before I dive into how to use the Spotify crawler, I want to say some things. A few things have made me really happy and thankful this week.

• My book got another five star review. It currently has 18 five star reviews, and nothing else. I really appreciate the enthusiasm of my readers! I don’t expect it to stay at five stars forever, but I’m enjoying it. If you’ve read my book, feel free to leave a review (even if not five stars). It helps other readers find the book.

• A few of my LinkedIn connections have been sending their networks to me and my blog. Thanks a ton. Network Science isn’t as popular as other topics in Data Science, and it takes a lot more energy to popularize this, so I appreciate it. I want people to learn this, because it is useful.

• Today, this blog has hit a nice milestone for 100th subscriber!

I’ve also been having some really great conversations with readers. A few interesting ideas have come up in our conversations:

• Once you start working with networks, it affects how you think. You no longer think in simple lists, checklists, and spreadsheets. You start to think about communities, ecosystems, flow, and how things influence other things. It gives you a deeper understanding of reality, once you see how things fit together and are able to dig into relationships and influence.

• Every single time we look at networks, we identify relationships that make us go ‘aha’. Like, they seem obvious after you see it, but not before. Your brain is building the connection, as well.

I just love talking to people who have read my book or work with networks, to hear their thoughts, and to learn how this awareness and understanding has affected their lives.

Thank you, everyone. It’s been a cool week.

Back to the Spotify Crawler

In 2020, I worked on a Graph Neural Network project for work. /we used a Scientific Paper Collaboration Network as the dataset. I often think about that project, and it made me wonder what a Music Collaboration Network would look like.

In 2021, I manually created a Science Fiction Author Collaboration Network, during the original #100daysofnetworks. That was really neat, and I’ll do that again, using crawling, eventually.

Today, I created a Spotify crawler, and it’s only current use is building Music Collaboration Networks, so that they can be analyzed.

For instance, some musicians collaborate with other artists. B.B. King was one such artist. I went to see him perform twice when I was younger, and I treasure those memories.

Other artists—such as Wilco—collaborate with other Musicians, but not as much.

And some artists, like Radiohead, don’t seem to collaborate with other Musician’s at all, at least not directly on their albums. But they do side projects.

Today’s crawler looks for direct collaboration, not side projects. It is also based on the Spotify API, which doesn’t seem perfect, but seems good enough to make interesting networks. In some of my searches, I noticed that it did not pick up artists that I know collaborated on an album. I can look into this, in future iterations of the crawler.

You can get the crawler code here.

And you can see the quick analysis here.

This crawler is brand new, so be patient with it. There might be bugs. It might not be perfect. The code is a bit gnarly, but I cleaned it up some. I have never crawled Spotify. Everything is new and confusing, but I did my best, and it seems to work.

What Do I See?

I have a few observations. If you use the crawler and set a seed artist who does not collaborate with anybody, then the network is going to be one node. Nothing worth looking at. If you don’t get anything interesting, it might not be a collaborative artist.

But other artists are much more interesting. B.B. King was a good one, and Steve Vai was another good one to use.

And you can pass in a list of artists, if you want to see how they overlap.

To build a decent network, I searched for B.B. King. Here is the whole network.

Without a doubt, this is cooler than many of the prepackaged networks that come with NetworkX. Learning how to build your own networks allows you to analyze the things that actually interest you, not somebody else’s data.

If I look at B.B. King’s ego network with a radius of 2, it looks like this:

And that led me to the questions:

• What did Lil Wayne do with B.B. King?

• What did U2 do with B.B. King?

Let’s find out!

What did Lil Wayne do with B.B. King’s music?

What did U2 do with B.B. King?

Networks have shown some collaborations I was never aware of.

This is so cool. I love this. Even if the networks are smaller than the Wikipedia crawler produces, they’re very interesting networks, and useful if you love music exploration.

I will experiment more and more with the spotipy library, as there is much more that can be done than this. Perhaps there are other networks and content that can be explored, using the spotipy library.

That’s All, Folks!

That’s all for today! Thanks for reading! If you would like to learn more about networks and network analysis, please buy a copy of my book!