Home / / Big data drives big demand for storage, IDC says

Big data drives big demand for storage, IDC says

Compound annual growth rate expected to be 53% between 2011 and 2016

Storage is a vital component of the big data and analytics infrastructure, remarked IDC.
Storage is a vital component of the big data and analytics infrastructure, remarked IDC.

As demand for big data technology and services continues to escalate, all levels of the big data technology stack will experience significant growth, according to market research company International Data Corporation (IDC).

Storage, as a critical piece of the infrastructure is expected to increase at a compound annual growth rate (CAGR) of 53% between 2011 and 2016.

IDC the recent publication of two in-depth studies: "Storage for Big Data: Insight Into Usage Patterns" and "Influencers in Deployment of Storage for Big Data". The reports are built on findings from the company's first-ever survey on storage infrastructure for big data and analytics.

The amount of data generated, processed, and stored by most organisations will continue to grow aggressively for the foreseeable future, IDC said.

"Storage will be one of the biggest areas of infrastructure spending for big data and analytics environments over the forecast period," said Ashish Nadkarni, research director, Storage Systems.

"Revenue from storage consumed by BD&A environments will increase from a mere $379.9m in 2011 to nearly $6bn in 2016. This growth will come largely from capacity-optimised systems, including dense enclosures. However, software-based distributed storage systems with internal disks to store post-processed data will also be embraced by some users."

Additionally, businesses will continue to tap into newer data sources as they move their analytics efforts from search to discovery. This shift will accelerate spending on infrastructure and data organisation platforms will continue to accelerate.

The "Storage for Big Data: Insight Into Usage Patterns" study assesses the results of IDC's Big Data Survey, conducted in the first calendar quarter of 2013, regarding trends in storage. Storage is a vital subsystem that can determine the success of a big data and analytics implementation. Capacity growth and application performance continue to be the top challenges facing organisations of all sizes as they relate to how storage is attached to big data and analytics environments.

Performance was cited as the primary driver for selecting storage architecture among 68.6% of respondents. Another 59.5% indicated cost as a primary driver (multiple responses were allowed). Just under 31% of respondents said they had no deployment of enterprise storage systems for data analytics infrastructure, but plan to start deploying in the next six months. The type of converged infrastructure deployed for big data infrastructure was split almost evenly between "discrete converged infrastructure" (30.1%), "compustorage" (29.4%), and "neither, we have done the integration in-house" (28.4%).

The "Influencers in Deployment of Storage for Big Data" report examines some of the qualitative and behind-the-scenes challenges faced by businesses with their big data infrastructure, either during or after deployment. IDC predicted businesses will continue to struggle with what data to analyse, how to store data before and after it is analysed, and how to feed the results of data analysis back into the business.

Analysis of operations-related data was cited by 63.7% of respondents as the primary use case for deploying data analytics infrastructure. Analysis of transactional data from sales or point-of-sale systems was cited by 53.3% of respondents. IT was by far the greatest influencer of data analytics infrastructure. Operations was a distant second. Improving customer satisfaction is the greatest business challenge to be solved with data analytics deployments, according to just over 61% of respondents.