Johan Fourie's blog

I'd rather be a comma than a fullstop

Can Twitter predict the markets?

with one comment


Ask anyone about the pitfalls of Twitter, and they might point to recent gaffs by a prominent South African politician as evidence that the dangers outweigh its benefits. But such warnings have not stopped many others, most notably the president of the United States, from tweeting on a regular basis: Twitter’s user base creates more than 500 million tweets a day, and it has added about 2 million new users in the last quarter of 2016.

Presumably this wealth of information must have some value. Twitter, sadly for its shareholders, struggles to turn such growth into profit: in the last quarter of 2016, revenue growth was only 1%. But because it captures public sentiment at a very granular level, it has attracted the interest of both scientists and entrepreneurs hoping to turn this information into public or private benefit.

The use of social media for prediction is, of course, not a recent phenomenon. Google Flu Trends, founded in 2008, used Google’s search engine to track the spread of flu in 25 countries. But excitement about the project waned as it struggled to make accurate predictions. A 2014 Nature paper noted the value of social media ‘Big Data’, but warned that ‘we are far from a place where they can supplant more traditional methods or theories’.

Twitter, though, seems to attract increasing attention. A 2014 paper uses Twitter to predict crime. A 2015 paper show how psychological language on Twitter predicts heart disease mortality. Another 2015 paper show how Twitter sentiment predicts enrollment of Obamacare. A 2016 paper show how Twitter could be used to predict the 2015 UK general elections.

But it is, understandably, the financial markets that has attracted the most attention. A 2016 paper by Eli Bartov (NYU Stern School of Business), Lucile Faurel (Arizona State University) and Partha Mohanram (University of Toronto) shows how Twitter can predict firm-level earnings and stock returns. They use a dataset of nearly a million corporate tweets by 3662 firms between 2009 and 2012, all tweeted in the nine-trading-day period leading to firms’ quarterly earnings announcements. The authors find, unsurprisingly, that the tweets successfully predicts the company’s forthcoming quarterly earnings, but find, more surprisingly, that the tweets predict the ‘immediate abnormal stock price reaction to the quarterly earnings announcement’. These findings are more pronounced for firms in weaker information environments, such as ‘smaller firms with lower analyst following and lower institutional ownership’, and are not driven by concurrent information from sources other than Twitter, such as press articles or web portals.

It makes sense that corporate communication provides information, but can public sentiment on Twitter also inform market activity? A 2017 NBER Working Paper, by Vahid Gholampour (Bucknell University) and Eric van Wincoop (University of Virginia), answers this question by looking at the Euro/Dollar exchange rate. They start with all Twitter messages that mention EURUSD in their text and that were posted between October 9, 2013 and March 11, 2016. There were 268 770 of these messages, for an average of 578 per day. What they hope to do, is to identify whether informed opinions about future currency changes can actually predict actual currency changes, so they eliminate all tweets that do not express a sentiment about the future behaviour of the two currencies. This reduces the sample to 43 tweets per day, or 27 557 in total. They then classify each of these tweets as positive, neutral or negative using a detailed financial lexicon that they develop to translate verbal tweets into opinions, and create a Twitter Sentiment index for each day. They also split the sample in two: those opinions expressed by individuals with more than 500 followers, which they call the ‘informed opinion’, and those with fewer than 500 followers, which they call the ‘uninformed opinion’.

So what do they find? It turns out that the 633 days of data they have is too short to calculate the Sharpe ratio, a measure of the risk-adjusted return. The annualized Sharpe ratio based on daily returns is 1.09 for the informed group and -0.19 for the uninformed group. The Sharpe ratio of 1.09 for the informed group is impressive, but it has a large standard error of 0.6. The 95% confidence interval is therefore very wide, ranging from -0.09 to 2.27. They then construct a model with a precise information structure, estimate the parameters and then recalculate the Sharpe ratio to average at 1.68 with a 95% confidence interval between 1.59 and 1.78. Success: ‘the large Sharpe ratios that we have reported’, they conclude, ‘suggest that there are significant gains from trading strategies based on Twitter Sentiment’.

If all this sounds terribly complicated, that is exactly the point. Translating opinions into numbers is not an easy undertaking, and discerning the ‘informed’ opinions from the noise is even less so. But there is no doubt that Twitter does offer some useful, perhaps even lucrative, insights. Whoever can exploit that knowledge first, stand to benefit most.

*An edited version of this first appeared in Finweek magazine of 20 April.


Written by Johan Fourie

May 16, 2017 at 06:24

One Response

Subscribe to comments with RSS.

  1. Very interesting blog post. Text analysis is exciting stuff. South-African banks and consulting companies are trying to exploit these useful information as well.

    Schalk Burger

    May 16, 2017 at 07:08

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: