Thursday, 9 October 2008

Last.fm, data mining and mashups

I've recently been putting together a Guide to Web 2.0 for The Helix Magazine and one of the most interesting aspects has been exploring the various mashups and applications of Last.fm.

Last.fm is a brilliant online music service and currently my favourite "web 2.0" application. By downloading a plugin for itunes (or whatever music player you have) that "scrobbles" each song you play (that is, tells Last.fm what you are listening to), a picture of your music taste builds up, and people with similar listening tastes are found. Artists are recommended to you according to your tastes, charts of your songs built up and "radio stations" perfectly tailored to you can be streamed online. But it is better than radio as there are no ads and you like every song.

By the way, I am westius on Last.fm.

Millions of songs are scrobbled every day by Last.fm users. This data helps Last.fm develop a massive database of user music preferences, and because of it's API, it is possible to access Last.fm information and develop interesting tools.

As users can tag their music with genres that they think aptly describe their songs and artists, it is possible to determine your own tag cloud of musical preferences. Using an excellent script at anthony.liekens.net, I came up with my own tag cloud, as you can see here.

It is possible from such tag clouds to examine how listeners fall into different categories through a process known as Data Mining. Data mining is essentially the process of sorting through enormous amounts of data and picking out the relevant stuff. Using principal components analysis - a mathematical technique which reduces multidimensional data sets to lower dimensions for analysis - and k-means clustering - an algorithm to cluster n objects into k groups - Liekens came up with 5 broad groups of Last.fm listeners:
  1. Electronic/pop
  2. Rock
  3. Indie
  4. Metal
  5. Hip-hop
Clearly this list does not reflect everyone on Last.fm (where are the classical music listeners?), but it does reflect the majority. I was surprised that Indie is a group in itself and am intrigued by the bundling of electronic and pop together - there are some tweaks to the maths you can make that could come up with different groups, and better results might be possible with a bigger data set . Hip-Hop listeners were the most clearly defined group. You can read more about the maths and how these groups are separated in the original article.

Another interesting thing you can do is compare your music tastes to your friends. This pic is a difference cloud comparing my music tastes with that of my good friend intranation. We have a roughly 40% similarity in music genre tastes, with the green tags those that I have more of in my collection, and the red those genres that intranation listens to more than me. No real surprises there.

Mashups are all the rage at the moment. The term refers to web applications that combine data from more than one source into a single integrated tool. For instance, domain, an Australian real-estate site, adds data from Google Maps to provide location information. My current favourite Last.fm mashup is idiomap. idiomap is a digital music magazine that personalises its content according to your interests in music, which it learns from your Last.fm profile. It gives you stories and reviews of the artists and genres you like, helps you discover new music and mashes in video and audio from youtube and other sources. idiomag aggregates music articles from over 100 different sources. You can also tweak the articles you like so if you receive something you don't like, you won't get it again. I subscribe to the RSS feed of my personalised idiomap magazine and so far its been great and has included reviews of music DVDs of artists I like and schedules of when bands will be playing and appearing on TV. Good stuff.

I will probably put out a few more blogs like this as I explore this world of mashups. And for podcast listeners, yes hopefully I will get one of them out soon too!

41 comments:

  1. Try to play and win with us.real online gambling only here Do not be afraid to play and take risks. Who does not risk that does not drink champagne.

    ReplyDelete
  2. Thanks for such a great article here. I was searching for something like this for quite a long time and at last I’ve found it on your blog. It was definitely interesting for me to read  about their market situation nowadays.
    Data science training in Bangalore
    Data science online training

    ReplyDelete
  3. ExcelR offers a mixed learning model where members can benefit themselves study hall, teacher drove online sessions and e-learning (recorded sessions) with a solitary enlistment. data science course in pune

    ReplyDelete
  4. Well, The information which you posted here is very helpful & it is very useful for the needy like me.., Wonderful information you posted here. Thank you so much for helping me out to find the Data analytics course in Mumbai Organisations and introducing reputed stalwarts in the industry dealing with data analyzing & assorting it in a structured and precise manner. Keep up the good work. Looking forward to view more from you.

    ReplyDelete
  5. I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it.
    machine learning courses in Bangalore

    ReplyDelete
  6. Such a very useful article. I have learn some new information.thanks for sharing.
    data scientist course in mumbai

    ReplyDelete
  7. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful.
    Data science course in mumbai

    ReplyDelete
  8. I am looking for and I love to post a comment that "The content of your post is awesome" Great work!
    Data science course in mumbai

    ReplyDelete
  9. Such a very useful Blog. Very interesting to read this article. I have learn some new information.thanks for sharing. know more about

    ReplyDelete
  10. The great information that you shared. It will help all of them. Thanks for posting. Keep maintain the updates
    PHP Development Company
    |
    ERP Solution
    |
    magento ecommerce
    |

    ReplyDelete
  11. Through this post, I know that your good knowledge in playing with all the pieces was very helpful. I notify that this is the first place where I find issues I've been searching for. You have a clever yet attractive way of writing.
    ExcelR Data Analytics courses

    ReplyDelete
  12. You might comment on the order system of the blog. You should chat it's splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!!
    Know more data science course in pune with placements

    ReplyDelete
  13. The great information that you shared. It will help all of them. Thanks for posting. Keep maintain the updates
    Green Pen
    |
    Ecommerce Solution Provider
    |
    Data Extraction Solutions
    |

    ReplyDelete
  14. Very nice blog here and thanks for post it.. Keep blogging...
    ExcelR data science training

    ReplyDelete
  15. Attend The Data Analytics Course in Bangalore with Placement From ExcelR. Practical Data Analytics Course in Bangalore with Placement Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analytics Course in Bangalore with Placement.
    ExcelR Data Analytics Course in Bangalore with Placement

    ReplyDelete
  16. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    ExcelR data science course in mumbai

    ReplyDelete
  17. nice information on data science has given thank you very much.
    Data Science course in Hyderabad

    ReplyDelete
  18. good article about data science has given it is very nice thank you for sharing.
    Data Science course in Hyderabad

    ReplyDelete
  19. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    ExcelR data analytics courses

    ReplyDelete
  20. Nice information, valuable and excellent work, as share good stuff with good ideas and concepts, lots of great information and inspiration, both of which I need, thanks to offer such a helpful information here. data science course

    ReplyDelete
  21. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    data analytics courses

    ReplyDelete
  22. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    data science course

    ReplyDelete
  23. I wanted to leave a little comment to support you and wish you a good continuation. Wishing you the best of luck for all your blogging efforts. I curious more interest in some of them hope you will give more information on this topics in your next articles.
    Data Scientist course

    ReplyDelete
  24. Hi,Thanks for sharing nice stuff...

    More: https://www.kellytechno.com/Hyderabad/Course/Data-Science-Training

    Data Science Training in Hyderabad

    ReplyDelete
  25. Goood information blog! Thanks for sharing please check our website about the Data Science Training
    Data Science Training in Hyderabad

    ReplyDelete
  26. Thanks for posting about the Data mining and mash up concepts explained ,please post like this type of articles
    Data Science Training in Hyderabad

    ReplyDelete
  27. I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end. I would like to read newer posts and to share my thoughts with you.big data course in malaysia
    data scientist course malaysia
    data analytics courses
    360DigiTMG

    ReplyDelete
  28. Truly, this article is really one of the very best in the history of articles. I am a antique ’Article’ collector and I sometimes read some new articles if I find them interesting. And I found this one pretty fascinating and it should go into my collection. Very good work!
    Data analytics courses
    data science interview questions

    ReplyDelete
  29. Interesting post. I Have Been wondering about this issue, so thanks for posting. Pretty cool post.It 's really very nice and Useful post.I am interested in some of them.I hope you will give more information on this topics in your next articles.
    Data Science training
    data analytics course
    business analytic course

    ReplyDelete
  30. Truly, this article is really one of the very best in the history of articles. I am a antique ’Article’ collector and I sometimes read some new articles if I find them interesting. And I found this one pretty fascinating and it should go into my collection. Very good work!
    data science course in bangalore
    data science interview questions

    ReplyDelete
  31. I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.
    ai training in bangalore
    Machine Learning Training in Bangalore

    ReplyDelete
  32. I am definitely playing your internet site.
    You really have a few terrific perception and remarkable testimonies.


    click here formore info.

    ReplyDelete