Wednesday 26 May 2010

Ep 130: Using Twitter to predict Movie Box Office Revenue (and the future...)

Ever wondered what good Twitter actually does? Personally, I love it, but really, is it anything but noise?

One of the pipe dreams for online social media is the ability to track opinions and interests in real time. In their paper Predicting the Future With Social Media, Sitaram Asur and Bernardo A. Huberman have not only tracked live opinion on movies, but used it to predict their future success.

Asur and Huberman, from the Social Computing Lab, HP Labs California, have shown that the rate of tweeting about a movie accurately predicts its opening weekend box office revene.

After examining the rate of chatter from almost 3 million movie tweets, the researchers constructed a linear regression model for predicting box-office revenues of movies in advance of their release. These results outperformed the Hollywood Stock Exchange, a market in which people can buy and sell virtual shares in actors, directors and individual movies and produces unusually accurate predictions of film popularity.

There is a strong correlation between the amount of tweets concerning a forthcoming film, and its opening weekend box office return. The rate of tweeting about a movie was determined by simply counting the number of tweets containing the movie name. The next step was to predict box office returns beyond the opening weekend, and this was achieved by including "sentiment" as a factor. Sentiment analysis is a fascinating area of linguistic study. Language classifiers were used to label the text associated with the movie tweet as Positive, Negative or Neutral. Adding these as factors into the regression significantly increased the researchers' ability to predict the box office returns beyond the opening weekend.

These results are intuitive - before a movie is released, potential viewers do not know whether they will like the movie and so simply the number of tweets about a movie gives an indicator of movie "buzz" and correlates with the number of people attending the opening weekend. Once a movie is released and people start forming opinions, movie tweets start to contain sentiment. Negative tweets, whilst they have little effect on the opening box office as no one has yet seen the film, have a strong influence on further returns. Likewise for positive tweets.

The question of cause and effect is very interesting. Does a high number of tweets about a movie actually cause a strong box office return, or are they correlated simply because the twitter and movie audience are arguably the same? Another way of asking this question is to ask whether an advertiser could change future box office returns by deliberately tweeting multiple times or with a particular sentiment.

I had a fascinating chat with Sitaram about this work. To listen to our chat, tune in here (or press play below):

Sitaram Asur, & Bernardo A. Huberman (2010). Predicting the Future with Social Media arXiv: 1003.5699v1

Wednesday 19 May 2010

Ep 129: The domestication of the dog and the Australian dingo

The Australian dingo and the New Guinea Singing Dog may be the world's oldest dog breeds.

The study, Genome-wide SNP and haplotype analyses reveal a rich history underlying dog domestication, which was published in Nature and is a major genetic study into the domestication of the dog, was a world-wide effort and had 37 authors - including Dr Alan Wilton, of the School of Biotechnology and Biomolecular Sciences at UNSW. The study found that the dingo and the New Guinea Singing Dog are the most closely related breeds of modern dogs to the original wolves from which all dog breeds come from. They are also the most like the original domesticated dog.

The study looked at 48,000 sites in the dog genome in hundreds of wolves, almost a thousand dogs from 85 modern breeds and several ancient dog breeds. Dr Wilton, a celebrated scientist in the field of dingo research and conservation who won the Unsung Hero of Science Award in 2004, provided the dingo DNA. The data suggest most dogs were domesticated in the Middle East around 10,000 years ago, rather than in Asia as had been previously thought.

Dingoes separated from other breeds of dog when brought to Australia around 5,000 years ago. However, they never made it to Tasmania, which became isolated from the mainland around 12,000 years ago. It is thought that the dingo out-competed the thylacine (Tasmanian Tigers) leading to its extinction on the mainland.

Other ancient breeds include Chow-Chow, Basenji, Akita, Chinese Shar-Pei, Siberian husky and Alaskan malamut.

I had a great chat to Dr Wilton about this work into the domestication - or as I repeatedly say, domestification... - of the dog and also about his work in dingo studies and conservation.

To listen to this show, tune in here (or press play below):

vonHoldt, B. (2010). Genome-wide SNP and haplotype analyses reveal a rich history underlying dog domestication Nature, 464 (7290), 898-902 DOI: 10.1038/nature08837

I want a Turbo Encabulator!

I also want some Prefamulated Amulite

Wednesday 5 May 2010

Ep 128: Another demotion for Pluto? Or is it about to become King of the Dwarfs?

Is Pluto is set to become an also-ran in the astronomical world?

Already demoted from the exalted planet club, Pluto could be joined by up to 50 other objects in the ever-expanding "dwarf planet" club if the new definitions of dwarf planet, recently proposed by Australian scientists Charles Lineweaver and Marc Norman, are accepted by the International Astronomical Union.

On the other hand, perhaps you would rather regard Pluto as the leading player in the astronomical second division. Rather than being the smallest of the planets, Pluto is set to become the charismatic king of the dwarfs.

The research, entitled The Potato Radius: a Lower Minimum Size for Dwarf Planets, suggests that the number of objects in the solar system classed as dwarf planets could grow by more than a factor of ten. One of the definitions of dwarf planet relates to its ability to exist in hydrostatic equilibrium - that is, the body must have sufficient mass for its self-gravity to pull it into a round shape. This is generally accepted to happen at a radius of around 400 km. Lineweaver and Norman found that the point at which a body loses its rugged and unshapely 'potato' appearance and becomes round depends on what the material that the body is made from. Icy objects form spheres at roughly 200 km radius, whilst rocky spheres form at 300 km. As these radii are considerably less than the generally accepted minimum radius for a dwarf planet - 400 km - the researchers believes that a whole new crop of trans-Neptunian objects should be classified as dwarf planets.

I had a fascinating chat with Dr Lineweaver about how they went about deriving these numbers, and also about their work defining the shape and mass of other astronomical bodies. To listen to this show, tune in here (or press play below):

Lineweaver derived these radii from first-principles physics, and compared the results to astronomical observation. Bodies are held together by gravitational and electronic forces, and the research showed a strong correlation between the mass and shape of astronomical objects. For example, at a radius of ~ 200 km - 300 km, moons and rocky asteroids transition from a rounded potato shape to a sphere.

The work also looked at other shapes in the Universe, and Lineweaver divided these into five basic shapes - dust, potatoes, spheres, disks and halos. Each of these shapes correlates with the size and mass of the object.

Most of the new dwarf planets will be out past Neptune – these are called trans-Neptunian objects. Pluto has a radius of 1,150 km, but it is not the largest known trans-Neptunian object - Eris has a radius of ~1,250 km.

The International Astronomical Union General Assembly, which makes the decisions re astronomical nomenclature, next meets in Beijing in August 2012. No matter what they decide, some people will always call Pluto a planet.