Nigel Sanctuary’s Big Data Blog 2, VP Cloud Propositions, Kognitio

Big Data, Cloud, Big Data, Cloud, Big Data, Cloud.  There, that’s the daily research reading done for analytical platforms and database technology. Now feel free to get on with your day.

What I mean is, is there anything new to say in this space? Well, perhaps. Can we talk about adoption? At Kognitio, we want to monitor who is addressing these issues and starting to build Big Data environments in the cloud. It’s hard to find out on the web and I am certainly not going to breach any NDAs by telling you who is looking at this end from our prospect and customer base. However, I think there are some views about adoption we can share.

In the technical space there are two fears that are delaying wholesale adoption according to Information Week and their November 2011 survey “The state of database technology”. These are virtualization of servers for scaling and how to manage security. For many, virtualization means a slowdown of performance in the database and with Big Data rapidly becoming a reality that is a real blocker. Security has so many different aspects it is hard to know which one to address first. “All of them”, I suppose is the simple answer.

Now it strikes me as obvious, that if Big Data is the subject of the moment and cloud-based technology means virtualization, you have a clash of cultures for a large portion of Database IT leaders. How are we going to sort that one out?

Amazingly, it’s actually quite simple. You find a database technology that embeds a function to virtualize hardware as a single machine and runs the database processing in parallel on this virtual environment. All you need to do is find a cloud-based provider who can offer this. Thankfully, Massively Parallel Processing on reserved instances of cloud-based computers is a reality; we have just such a service here at Kognitio. What is rarer, is one that enables the performance to be maintained with the virtualisation in operation. Most virtualized, cloud-based services don’t parallelize the database across the virtual machine environment making a single instance of the database platform from multiple servers. And, only one of them does that in-memory…in the cloud (Guess Who?)

The statistics were the part of the Information Week report that interested me. Apparently 37% have already adopted a virtualized database environment and 23% were going to in the 24 months from last September or thereabouts. Great, I thought, a ready-made market. But alas, it appears that only 7% are willing to put this architecture in the cloud. That will be the security issues, I imagine. Solvable, easily solvable, but we understand the caution. It’s a very emotive issue.

So where do we go from here. The desire is there, the platforms are there, the problem is here. Where’s the catalyst?

It’s a bit like watching those penguins coming ashore in Australia. You know the ones, where two or three penguins rush up the beach to test the safety and then rush back in to the sea. Only when this ritual has been performed a few times does a brave penguin stay and the rest rush up the beach together.

In the technology world, the equivalent of rushing up the beach is trying the demos and proof of concept facilities offered by Kognitio, for example, before settling down to lay your eggs…sorry, got carried away with my analogy there.

So come on, which of you is going to be the first penguin?

Nigel Sanctuary
VP Cloud Propositions, Kognitio