IDC: Hadoop commonly used with other big data analytics systems

IDC: Hadoop commonly used with other big data analytics systems

Enterprise Hadoop deployment survey reveals complex picture

Article comments

A study by analyst IDC shows how companies are using the open source Hadoop big data analytics systems alongside other systems to get value out of their data.

IDC's “Trends in Enterprise Hadoop Deployments” report, commissioned by Red Hat, found that 32 percent of companies questioned had deployed Hadoop. An additional 31 percent said they had plans to deploy Hadoop within 12 months, and 36 percent said their Hadoop deployment schedule could go beyond 12 months.

The study found that enterprises are combining Hadoop with other databases for big data analysis. Nearly 39 percent of respondents said they use NoSQL databases like HBase, Cassandra and MongoDB, while nearly 36 percent said they used MPP databases like Greenplum and Vertica.

"This situation underscores the importance of causality and correlation, in which traditional structured data sets are analysed in conjunction with unstructured data from newer sources,” the report says.

The report confirms the point made by Facebook analytics chief Ken Rudin earlier this week when he told a New York conference that Hadoop was not enough for organisations looking to exploit big data.

The IDC study shows the various ways companies are using Hadoop. These include the analysis of raw data, whether it is operations data, data from machines or devices, point of sale systems or customer behavioral data gathered from ecommerce or retail systems.

Some 39 percent of respondents said they use Hadoop for "service innovation", which includes the analysis of secondary data sets for modeling of "if-then" scenarios for products and services.

Some of the less popular use cases for Hadoop include its deployment as a platform for non-analytic workloads, for example, in conjunction with a SQL overlay for OLTP (online transaction processing) working.

As a result, said IDC, enterprises are looking to alternative persistent storage systems. According to the report, “File systems like IBM’s Global File system (GPFS), Red Hat Storage (GlusterFS), EMC Isilon OneFS and others that have earned a reputation for their robust scale-out capabilities, are clearly preferred as alternatives to HDFS (Hadoop Distributed File System)."

The survey also found that most enterprises process big data both before and after Hadoop processing. "This highlights another attractive feature of other storage alternatives, including the ability to keep the data in native POSIX format and use traditional analysis tools," said IDC.



  • ChrisBrown_OCF Thought-provoking article Firstly Im assuming the 32 per cent of companies the survey claims are deploying Hadoop are relatively large organisations I really cant imaginesuch a high percentage across a broad spectrum of companies working with the framework It would be interesting to know the respondent demographics behindthose stats Second it really isnt a surprise that companies deploying Hadoop are combining it with databases and file systems It is essential Hadoop is just a framework It needs these additional tools to function However I am surprised though that companies arenot persisting with HBase which comes as part of theHadoop distribution and HDFS If a business is showing interest in Hadoop then it would be safe to assume that it is comfortable with the whole concept of open source non-proprietary tools Why would it then swap out HBaseand HDFS for tools that take away those freedoms My recommendation to businesses looking at Hadoop would be to always try it out first before swapping in more costly and proprietary tools And if you are going to financially invest in a Hadoop architecture then do so with companies who addincremental services or products to Hadoop and not replacement technologies
Send to a friend

Email this article to a friend or colleague:

PLEASE NOTE: Your name is used only to let the recipient know who sent the story, and in case of transmission error. Both your name and the recipient's name and address will not be used for any other purpose.

We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message
* *