Currently many organisations who deal with data have amassed more information than they can easily store or manage. Unfortunately, the rate of data growth will only continue to accelerate. Data analyst, Wikibon, forecasts that on the back of a super-charged compound annual growth rate (CAGR) of 58 percent between now and 2017, the market will top the $50 billion mark. Yet, these figures only tell part of the story. Today, rather than just amassing big data, companies increasingly want to ‘do things’ with it: they want to find ways to process, analyse and mine data to enable them to make more informed business decisions.
The good news is that due to technological advancements in this market, big data is becoming more democratic, enabling smaller organisations with more limited resources to utilise these technologies and compete on a more even keel with their larger enterprise peers and start to use information to drive agility. However, to turn theirvision into a reality and start to deliver the commercial benefits they are striving for, SMEs need to be working in an open-source environment.
Open-source technology, now aided by cloud computing, has always been in the vanguard of massive-data management. In fact, the rate at which the importance, popularity and usefulness of big data has grown runs parallel with the growth and popularity curve of open source software. Open-source software is clearly leading innovation in the big data space.
The leading big data platform, Hadoop, is an open-source framework for processing and analysing the vast data volumes companies generate every day. For SMEs who have historically struggled to find the resources they need to effectively manage Big Data, Hadoop offers the most effective way to address the challenge and get the job done faster and more accurately. Hadoop and other open- source technologies provide a flexible and scalable architecture that can manage large volumes of data cost-effectively – a critical consideration for SMEs. They are especially effective in this context, when coupled with open-source data management and integration software solutions from vendors like Talend which help smaller businesses process, analyse and interpret the data.
From Theory to Practice
Currently we are seeing a wide range of different use cases in which the implementation of open- source technologies is helping democratise big data and open it up to SMEs. One example could be smaller businesses capitalising on online recommendations in order to influence a customer’s buying decisions. Large organisations like Amazon and eBay have used recommendation engines for years now to perform tasks like matching and recommending products, people and advertisements to users. Now, because open-source technology makes massive-data management more accessible, smaller organisations can benefit from the processing power of technologies pioneered by these giants, without the need for extensive investment or in-house expertise.
Customer retention is another area where open-source and Big Data are working in tandem, helping to level the playing field and enable small businesses to compete with their larger peers. Large organisations have for some time used complex techniques to measure the likelihood that customers may ‘churn’, based on profiling against other customers and realistic appraisal of the competitor’s attractiveness. Now the latest open-source technologies are making this kind of capability accessible to SMEs too.
The emergence of new open-source solutions is significantly bringing down the cost of Big Data initiatives and helping to democratise the whole Big Data arena. There is little or no upfront cost as open-source software was designed to run on commodity hardware. From the perspective of SMEs, open-source technologies are allowing them to gain a level of intelligence that would have been completely out of their reach before.
Hadoop and other open-source cloud computing technologies are driving this change. When coupled with open- source data management and integration solutions they are helping to lower the technical barrier to Big Data solutions. Today they are enabling even the smallest enterprise to get to grips with their Big Data issues, answer complex operational questions cost-effectively, improve decision-making capability and ultimately achieve the holy grail of competitive advantage over their larger business rivals.
Talend are a gold sponsor at this year’s Whitehall Media Enterprise Cloud Computing & Virtualization conference, to be held at the Hotel Russell, London, on 7th March.