Social Media Lead Mining Company Essencient Proves Its NLP Tech By Predicting EU Referendum To Within 0.3%

Social Media Lead Mining Company Essencient Proves Its NLP Tech By Predicting EU Referendum To Within 0.3%

Essencient today announced that its Social Media Lead Mining technology, which uses patented Natural Language Processing (NLP) technology to remove noise around a chosen subject or brand, had proven accurate enough at mining meaningful posts on Twitter to be able to predict the UK EU referendum to within 0.3%.  The company said that as far as it knew it was the only commercially available platform to get this close. Essencient’s results validate the potential of its new and innovative approach to accurately mining actionable data, such as sales leads and customer support requests, from social media.

The company processed posts from Twitter on the EU Referendum and analyzed sentiment for the period of 7 days up to midnight of the day of the vote, and compared these findings against the actual results. The results were -0.3% off the actual referendum result, with their analysis predicting that ‘Leave’ would be the winning result. Additionally, they plotted data during the run up and sentiment was indicating a ‘Leave’ win for most of the two weeks leading up to the vote.  They also performed a comparable analysis for Twitter data from the London Mayoral Election that took place in May, and their results for sentiment were +2.7% off the actual second round voting result. When they analyzed both campaigns for ‘intention to choose’ the variances were -1.5% and -0.6% respectively, again both clearly indicating the actual outcome.

The Essencient platform takes a novel approach, using its NLP technology to analyze texts from sources such as Twitter and identifying the ones that have no meaningful linguistic markers in relation to a chosen subject. These are then removed, to leave a stream of high quality, meaningful conversations with markers such as sentiment, intent, seeking/giving guidance or flamboyancy of writing style, all indicators of engagement. These meaningful conversations can then be reviewed for action or analyzed for insights.

Essencient COO Mike Petit said, ‘Over the sixteen years I have been involved with NLP and social media, the amount of noise has become the most serious hurdle to effective engagement. By noise we mean the number of posts where a target such as a brand might be mentioned, but nothing meaningful is said about it. Because we eliminate the noise, humans don’t have to wade through huge amounts of raw posts to find the good stuff, and the analyses and decisions based on cleaner data are much higher in quality. Think of it as reversing the process of looking for needles in a haystack. The hay is much easier to see, so why not find that and get rid of it to leave just the needles? Since the noise can be as much as 95% of the data, it saves a lot of effort and delivers a lot more value.’

When asked why no one else was doing this, Petit responded, ’Our patented NLP tech is unique in what it can identify. For example, no one else is able to identify if an author is seeking or giving guidance like we can. We can also find references to a topic across sentences when pronouns like “he”, “she” or “it” are used to tie them together, a process called co-referencing that is on the cutting edge of computational linguistics. These and other advances permit our tech to deliver levels of accuracy that are unmatched.’

Essencient CEO Rob Lancashire added, ‘The analysis of the EU Referendum and London Mayor Elections was an exercise to demonstrate the accuracy of the system, and whilst it validates the market leading ability of the system, predictive analytics is just one aspect of the real value our platform delivers, which comes from helping brands to increase engagement with customers on social media. Consider that 48% of messages towards brands are not replied to, and that the average response time when a brand does respond is 9 hours. Ad to that the fact that only 3% of posts are directed to the brand using its handle or tag, and it’s clear a lot more can be done. But it’s not easy. Social media etiquette typically requires a brand to intervene in a conversation only when addressed, and not to intrude when only mentioned in passing. The accuracy of our platform covers all these points for brands by delivering only the posts that are meaningful, so a brand can efficiently and quickly make engagement decisions case by case. This could be a sales opportunity or a customer support issue, for example, but either way when you consider customers are reported to spend between 20-40% more if they engage on social media, there is a big opportunity out there waiting.’

About the Author