Getting Data Centers Ready for Big Data
Big Data has become a popular term for the vast amounts of data that is flowing into our data centers with increasing velocity and variety. The Internet of Things (IoT) constitutes a growing part of Big Data as it arrives in terabytes or exabytes (1,000,000 TB) per data set.
Since IoT and Big Data arrive at a higher velocity than traditional data, it is not only more difficult to analyze, but can also put additional stress on data centers. Where traditional data is highly structured, data from the IoT will not be nicely packaged and will likely be unstructured in nature.
How do IoT and Big Data affect the data center?
The research firm Gartner, Inc. believes that the IoT will include 26 billion units sending data to be processed by 2020. They also expect that the product and service supplier market for IoT will create revenue exceeding $300 billion in the same timeframe. According to Gartner, “The Internet of Things (IoT) has a potential transformational effect on the data center market, its customers, technology providers, technologies, and sales and marketing models.”
All of this data will need to be processed and analyzed which will increase the workload for data centers, forcing them to deal with new capacity, security, and analytics problems.
Joe Skorupa, vice president and distinguished analyst with Gartner said, “The enormous number of devices, coupled with the sheer volume, velocity and structure of IoT data, creates challenges, particularly in the areas of security, data, storage management, servers and the data center network, as real-time business processes are at stake. Data center managers will need to deploy more forward-looking capacity management in these areas to be able to proactively meet the business priorities associated with IoT.”
The Internet of Things is going to have an impact on storage management practices in data centers across the world. These data centers will need to radically increase infrastructure resources and storage capacity to be ready to handle the Internet of Things-related data.
Bandwidth will also be affected in both businesses and data centers. The Internet of Things is going to shift the trend away from old lower speed WAN links to higher-speed connections that are able to handle all of the small messages coming from the devices that comprise the IoT.
Data centers must start preparing for the additional storage, processing, and bandwidth required for Big Data and the Internet of Things. Of course, in order to handle this additional load, data centers will also face challenges with increased virtualization, which requires additional power and cooling to handle the additional load.