The convergence of Big Data with analytics: Microsoft UK3


How can businesses use Big Data analytics effectively in an enterprise data environment? That was the question posed by Ryan Simpson of Microsoft at this year’s Whitehall Media Big Data Analytics conference. Ryan, a technical solutions professional working with emerging technologies, argued that the biggest challenge facing enterprise at the moment is to accept that they have to adopt and embrace new techniques and technologies if they want to harvest and share valuable business insights that will help to drive business growth. Any business that fails to move with the times will only fall further and further behind their competitors.

Data Needs: volume, velocity and variety

Big Data Analytics is now cool according to Ryan. More and more people are now getting access to large amounts of data, and are realising that large volumes of processed data have the capability of delivering business insight which can help to drive profitability. However, the real value lies not just in the amount of data, but in the way the information is analysed and applied, in the breadth of the data, and in the speed at which it is processed.

Volume is important, and it’s also changing. However, volume isn’t an end in itself. Large volumes of processed data can deliver insight, but they also unfortunately drive more data. This extra data then also has to be processed to get insights. Velocity is also changing. The time frame from processing the data to turning it into useful insights is getting significantly quicker. New advanced technologies mean that we are now capable of producing programmes that can process massive amounts of data from a live website and turn that information into a targeted advert on the site on record time. The variety of new information has also changed the way we analyse the data we receive. Insight-derived analytics is now to a large extent self-fulfilling. We have a clear idea of what insights we would like to find, and are now sufficiently advanced to be able to only process the data that will produce the insights we require. With these insights enterprises can then revisit the data to get more advanced analysis. That lifecycle has a direct effect on both volume and velocity.

Why is advanced analytics crucial for enterprise?

The IDC’s Digital Universe Study which was sponsored by EMC threw up some startling statistics:

  • The total global rate of big data has grown by 60 percent at a compound annual growth rate, effectively creating a tenfold increase every 5 years. (Volume)
  • 85 percent of this big data comes from new data sources, like video, Twitter feeds, Facebook status posts and geo-location data. (Variety)
  • The cost of storing big data has fallen by nearly 17 percent year on year since 2005. (Velocity) With technologies like Hadoop we can now process significantly more data than ever before at remarkable speeds. The costs are likely to continue to fall.

However, this presents something of a double-edged sword. Not only do enterprises have to store this burgeoning amount of information: they have to process it too if they are to gain any valuable business insights. It’s up to IT departments to store, scale, manage and govern the data, and that can be onerous. The challenge, therefore, is to accept and adopt new tools and techniques which will allow them to harvest this valuable data.

Microsoft argues that every business will face its own Big Data challenges over the next few years. How it responds to the growing volume and variety of data and how quickly it can analyse it will determine how successful it will be. Ryan Simpson cited Yahoo as an example of using Big Data effectively. Yahoo doubled its revenue stream by allowing campaign managers to have access to the same data as the algorithms, and allowing them to use this data to guide the algorithms to focus on the insights they required. This upshot was that not only did targeted advertising increase growth, but that the company massively increased its revenue from advertising.