Thursday, August 05, 2010

Predicting Stock Market Indicators Through Twitter “I hope it is not as bad as I fear”

We have been working on trying to predict market indicators for quite some time by analyzing Web Buzz, predicting who will win an Oscar, or how well movies do at the box office. Among other things we have correlated posts about a stock on Yahoo Finance and Motley’s Fool with the actual stock price, predicting the closing price of the stock on the next day based on what people say today on Yahoo Finance, on the Web and Blogs about a stock title.

The rising popularity of twitter gives us a new great way of capturing the collective mind up to the last minute. In our current project we analyze the positive and negative mood of the masses on twitter, comparing it with broad stock market indices such as Dow Jones, S&P 500, and NASDAQ. We collected the twitter feeds from one whitelisted IP for six months from March 30, 2009 to Sept 4, 2009, ranging from 5680 to 42820 tweets per day. According to twitter this corresponds to a randomized subsample of about one hundredth of the full volume of all tweets, as the total volume in 2009 was about 2,5 million tweets per day. We tried to measure collective hope and fear on each day by applying the simple metric of counting all tweets containing the words “hope” – there were 54 to 467 tweets per day, and “fear” or “worry” – there were 9 to 100 tweets per day. This tells us that people prefer optimistic words (hope) to pessimist words (fear or worry).


As external benchmark of investor fear we used the Chicago Board Options Exchange Volatility Index VIX, which is strongly negatively correlated with Dow, S&P 500, and NASDAQ, which is not surprising, as the spread of stock options on a given day is used to calculate VIX. Initially we expected the number of tweets with hope to negatively correlate with VIX, and the number of tweets with fear or worry to correlate positively with VIX. Surprisingly, we found positive weak but insignificant correlation for both “hope” (0.135) and “fear” or “worry” (0.172) with VIX, and negative significant correlation with both “fear” and “worry” and “hope” with Dow NASDAQ and S&P500 (This means that people start using more emotional words such as hope, fear, worry in times of economic uncertainty. We therefore created a simple twitter-volatility index combining mentions of hope, fear and worry, normalizing it with the total amount of tweets per day as a baseline. This index displays strong significant negative correlations to Dow, NASDAQ and S&P500, and strong significant positive correlation to VIX (see table below).


The picture below visualized the negative correlation between Dow (blue) and “hope, fear, and worry” (green) in the period March 30, 2009 to Sept 4, 2009.


To put this in simple words, when the emotions on twitter fly high, that is when people express a lot of hope, fear, and worry, the Dow goes down the next day. When people have less hope, fear, and worry, the Dow goes up. It therefore seems that just checking on twitter for emotional outbursts of any kind gives a predictor of how the stock market will be doing the next day.

Just to be clear, what we have presented here are very early preliminary results, and much more work is needed to scientifically verify it.

12 comments:

  1. Your article was very nice...yes the prediction from twitter is never bad

    ReplyDelete
  2. I really appreciate it and you have shared very valuable blog. We can utilize ti and i would like to thank to you a lot for sharing this.

    ReplyDelete
  3. I really appreciate your post and you explain each and every point very well.Thanks for sharing this information.And I’ll love to read your next post too.

    Money Maker

    ReplyDelete
  4. This comment has been removed by a blog administrator.

    ReplyDelete
  5. Just want to say your article is as amazing. The clearness in your post is simply cool.
    -------------
    stock trading strategy

    ReplyDelete
  6. This is so cool. I am such a huge fan of their work. I really am impressed with how much you have worked to make this website so enjoyable.

    ReplyDelete
  7. Nice Information! I personally really appreciate your article. This is a great website. I will make sure that I stop back again!.Stock Tips

    ReplyDelete
  8. Useful information shared. I am very happy to read this article. Thanks for giving us nice info. Fantastic walk through. I appreciate this post. stock tips

    ReplyDelete
  9. Nice Information! I personally really appreciate your article. This is a great website. I will make sure that I stop back again!. stock tips

    ReplyDelete
  10. This is so cool. I am such a huge fan of their work. I really am impressed with how much you have worked to make this website so enjoyable. best advisory company

    ReplyDelete
  11. I always like your blog post because you always comes with different ideas and information. I always shared your site post with my friends. Keep posting and i will follow you. Stock tips

    ReplyDelete
  12. I feel strongly about this and I take pleasure in learning about this topic. If possible, as you gain data, please add to this blog with more information...

    ReplyDelete