Data mining: Forget big data, what we need are big answers

What’s more important: the amount of data you manage to collect, or what you do in terms of data mining? Is it better to gather as much information as possible, or to use a smaller amount of information more effectively? Well, according to Harper Reed, CTO of President Barack Obama’s re-election campaign, it’s not the big data that’s important; it’s the big answers that data mining gives you. In short, effective analytics are the key to success and the only way to gain true and accurate insight.

The value of data mining is underrated

In a speech at the EMC Greenplum launch of Pivotal HD platform, Reed put the current debate about the true value of big data into perspective. He told delegates that big data isn’t really all that important in itself. The people who supply the money for big data projects aren’t looking for a solution to the big data problem. What they want are answers – and not just any old answer. They want “big answers: big answers to those big questions that will allow them to advance their companies and their careers and give them a competitive edge. At the end of the day, Reed claimed it doesn’t matter to these companies whether the data used is big, medium or small: they simply want answers to critical questions and they want those answers “yesterday.” A high level of data mining can provide this.

He highlighted the essential difference between the Republican and Democratic 2012 election campaign as proof of that. Romney outsourced his analytics to a number of expert data mining firms that didn’t have much political knowledge, whereas the Democrats handpicked a team that focused on the specific problem, which was identifying the best way of getting re-elected. As a result the Republicans focused on the election result and used their analytics to prove they were ahead, whilst the Obama campaign focused on the achieving the desired result and used data mining to assure it would win.

As a result, the Democratic campaign was much more effective at not only seeing where the race would end up but, through effective micro targeting, in assuring a win. It was able to use the email addresses and social network IDs of Democratic candidates and their spouses to more effectively raise cash and to get the vote out.

More emphasis is needed on data mining, rather than storage

According to EMC’s Vice President of Quality, Jim Bampos, Reed’s approach is broadly similar to EMC’s approach to big data analytics. He argues that whilst the executive funding any big data effort may understand that big data is important, he or she does not find the process of storing, managing, protecting and optimising that data particularly interesting. What the executive really wants are timely answers that will help his or her company win in the market and advance their careers. If the focus is purely on the collection and storage of the data itself, this ‘win’ will not be achieved. What will bring success is giving the executive information he or she can use effectively, after data mining productively. Providing these critical and timely answers was the reason Barack Obama won an election he was expected to lose. That’s why EMC and Reed maintain that it’s only big answers, and not big data that will assure businesses succeed in a congested and highly competitive marketplace.