Sunday, 15 September 2013

Modelling an all-time greatest musical playlist

The popularity of Triple J's annual Hottest 100 has made my wonder what my favourite songs of all time are and whether I could come up with a list based on some actual data. The information I have to use is my iTunes data since 2005. Being only 8 years of my life, this data set is limited. But with any luck (that is, if the assumptions hold true) the following algorithms will stay appropriate into the future and require only minor tweaking. What we're trying to do is come up with a method that will tell me, from my listening habits in iTunes, what my favourite songs are. Whether you actually listen to your favourite songs more than others is a debate for another time.

iTunes doesn't tell you when songs were played, just how many times, so the useful parameters we can export for each song are "Play Count" (p) and "Date Added". If we add up all the individual play counts, we get the "Total Play Count" for the entire collection (P). Date Added can be turned into the number of days the song has been in the collection - time (t). We also know the number of songs in the collection now (N) and at various times in the past when I've exported the data.

First cut:
An easy first-cut model is to simply divide each song's play count by its time in the collection and order the songs by this rate of play. As a first attempt this may seem logical, however the problem is that it is heavily biased towards newer songs. You're likely to listen to a song a few times after you add it before it slips back into your various playlists. It also doesn't take into account that there are more songs in the collection now than at the start.

What we need to do is come up with an equation that tells us how many times a song is expected to have been played depending on when it was added. We can then compare this number to how many times it was actually played and order the songs by this ratio.

Second cut:

This second version suffers from the same biasing problem as the first, but does take into account that the number of songs in the collection is changing over time. This is important as if you assume that you listen to music for about the same amount of time each day, then the more songs you have in your collection, the less likely you are to randomly hear the same song twice. Hence, songs that are played regularly when the collection is small should not be treated in the same way as songs played with the same frequency when the collection is large. N0 is the number of songs in the collection at t0. This model assumes that the number of songs in the collection grows linearly over time (A and B are constants) - that is, the same number of songs are added each month. This is about right for my collection. The integration is left as an exercise for the reader (hint, you get a log function).

Third cut:

This final version takes into account that when you add new songs to your collection that you like, you are likely to listen to them quite a lot, independently of the number of songs that are already there - that is, they get added to a "new songs" playlist. The novelty of a new song eventually wears off, so the way we've modelled this is to use an exponential factor. You can tweek the coefficients (C and D) by thinking about the "half life" of a new song. The integration is left as an exercise for the reader (hint, you get a log function and an exponential).

The equation now contains two components - the first modelling the number of plays expected through random play and the second the impact of adding new songs to the collection. The model suggests that I play the same number of songs each year (apart from a barely perceptible increase due to the exponential factor) and it seems to work pretty well. This model won't work if and when I swap over to streaming music, as opposed to owning it, as my major form of music consumption, but for now it's holding up. Having played around with the coefficients, the list as it stands is below. It pretty much represents upbeat songs I go running with and songs my 2-year old likes - for whatever reason, he likes Korean pop music! I have to think that the novelty of Psy will wear off over time, but Hall and Oates, they'll never die.

Gangnam Style PSY
I Remember Deadmau5 and Kaskade
ABC News Theme Remix Pendulum
You Make My Dreams Hall & Oates
Shooting Stars Bag Raiders
This Boy's In Love The Presets
Get Shaky Ian Carey Project
From Above Ben Folds
Banquet Bloc Party


  1. Toblay movie songs is really a best web site for latest songs as well as movies use this web site from many days. Greatest website for song lovers, here you can find latest audio songs also old songs. This website service very best. You can download instantly songs

  2. Your blogs and every other content are thus interesting and helpful it makes me return back again.

  3. Quickly this site will indisputably be famous among all blogging people, because of its fastidious articles or reviews.student discount at foot locker

  4. Your blogs and every other content are thus interesting and helpful it makes me return back again.cordless vacuum cleaners

  5. Keep it up!! You have done the nice job having provided the latest information.billboard advertising costs uk

  6. Cool blog site friend I'm about to suggest this to all my listing cleaners

  7. I don’t waste my free time that’s why I read the informative things when I got this blog I really enjoyed reading could try here

  8. Hey buddies, such a marvelous blog you have made I’m surprised to read such informative stufcheck these guys out

  9. I know this is quality based blogs along with other stuff. criminal lawyers

  10. I guess this blog is perfectly company

  11. If you should be opting for finest contents like me, just visit this blog site daily because it provides the feature contents, thanks.Mark Curry payday loan

  12. I feel happiness to read the content that you are

  13. Thanks for sharing out such a great informative content are extremely fastidious.
    tubemate download

  14. It is a life that would become centered around a great mystery of how the musical talent would blossom into a recognized and celebrated gift; a life that would alter the musical landscape and the spiritual worship realm in a short 24 days, and a life that would become so influential that it would dictate musical compositions for many years spotify follower

  15. Master the Mind know what the image means; it no more has the power to override our feelings and measures. This is not because we now have overpowered it, but rather due to the fact we now have included the training or meaning powering the graphic.

  16. You have really selected the suitable topic; this is one of my favorite blogs.

  17. There are numerous types of genitals pants as well as vaginal area briefs. Largely Schamlippenverkleinerung, the alluring pussy is constructed of soft, meats-like Sylikon. For a lifelike sensation to the wearer. Guy just slides in, puts his thing in the genitals and off you chooses to go. The penis is already so inside the pussy how the glans to the clitoris displays. So if person desires to inspire his best item, he just must experiment around the clit

  18. Boy that’s some hardcore quantitative approach. I, however can remember the most played songs in that time on tips. These songs take you back in time like nothing else. Also I cannot believe Gangam Style is this old. تجميل