Twitter mood predicts the stock market with 86.7% accuracy
Some people think Twitter is about sharing pointless little messages or about self promotion. However, there’s more to Twitter than this! Whilst a study by Pear Analytics in 2009 estimated that 40% of Twitter messages are just 'babble', this means that 60% isn't. And you can take advantage of that.
You can use Twitter to share but also to search and extract information; like giving and taking. When a lot of people share a lot of messages on a daily basis you’ll get an enormous amount of data. You can use smart computer algorithms to analyze this data and create information from it.
‘Twitter mood predicts the stock market’ [pdf] is the headline of a scientific paper published in the Journal of Computational Science in March 2011 by Johan Bollen et al. Their research is relevant since the main conclusion is that mood states, but not sentiment, can improve the accuracy of predictions of the Dow Jones Industrial Average (DJIA).
What is the difference between sentiment and mood states?
Sentiment is mood measured as positive, neutral or negative based on emotionally charged words.
A more sophisticated method of measuring the mood of the public is via a standard psychology tool called the Profile of Mood States (POMS). This is a questionnaire that classifies people’s feelings into six categories: Tension-Anxiety, Anger-Hostility, Fatigue-Inertia, Depression-Dejection, Vigor-Activity and Confusion-Bewilderment. This instrument is widely used for measuring and monitoring treatment change in clinical and medical centers.
Can one really measure public mood states using Twitter?
To confirm that public mood states can be measured in Twitter Bollen et al. looked at the public mood on Election Day 2008 and Thanksgiving.
The mood calmness dropped the day before the election corresponding with pre election anxiety and returned to normal on the day itself and after. On Thanksgiving the mood state ‘happiness’ peaked as you might expect. These results are in line with common sense so the conclusion is that the public mood can be measured via Twitter.
Why refer to the public mood state to predict the stock market?
Can the stock market be predicted? According to the Efficient Market Hypothesis stock market prices are largely driven by news rather than past prices. And since news is essentially unpredictable, stock market prices will follow a random walk pattern which can only be predicted with a 50% accuracy. However this appears to not be the case.
Stock market prices don’t follow a random walk and can be predicted to some degree. Also, news may not be as unpredictable as we think. In addition, emotions and moods play an important role in human decision making. Ever heard the saying that stock markets are driven by fear and greed? This led to the hypothesis that public sentiment can be used to predict the stock market.
Inclusion of public mood state increases prediction accuracy
Bollen et al. used a computer algorithm and 9.7 million tweets to predict whether the stock market would go up or down. Using data from previous days they could predict the shape of the stock market with 73.3% accuracy. Upon inclusion of POMS data their prediction accuracy went up to 86.7%.
Important to note is that while POMS data enhances predictability, public mood states in terms of positive versus negative mood did not. Especially the calmness of the public was predictive of DJIA shifts that occurred 3 to 4 days later.
Now whilst this is a great finding opening up a wide range of possibilities, it also raises questions that need answers. Future research must be done to elucidate the mechanism (how can the calmness index be correlated with the stock market) and reproduce the result in different areas and ideally in real time.
Meanwhile we have BuzzTalk in place for you to analyze twitter feeds, but also blogs and news sites for both sentiment as well as mood states based on POMS.