Big data gap: Only .5% of data is analyzed

There is an untapped big data gap. Only .5 % of data is currently analyzed by companies to help them make decisions. IDC’s latest Digital Universe Study sponsored by EMC showed that in 2012, 23% of the information in the digital universe (or 643 exabytes) would be useful if it was tagged and analyzed. However, only 3% of the potentially useful data is tagged and even less — only .5% — is analyzed.

The newly released survey shows that the vast majority of new data being generated is unstructured. As a consequence, companies know very little about this data unless it is somehow characterized or tagged—a practice that results in metadata, which is one of the fastest-growing sub-segments of the digital universe.

IDC estimates that by 2020, as much as 33% of the digital universe (more than 13,000 exabytes) will contain information that might be valuable if analyzed, compared with 25% today. This untapped value can be found in social media usage patterns, correlations in scientific data from discrete studies, medical information intersected with sociological data, faces in security footage, and so on.

The survey forecasts the digital universe will be generating 40 zettabytes by 2020. Last year’s forecast for 2020 was 35 ZB. This raises the question: how can companies protect this amount of data? The survey showed that the proportion that will need some sort of protection will increase from less than a third in 2010 to more than 40% in 2020.

Other survey highlights:

• From 2005 to 2020, the digital universe will grow by a factor of 300 from 130 exabytes to 40,000 exabytes, or 40 trillion gigabytes (more than 5,200 gigabytes for every man, woman and child in 2020). From now until 2020, the digital universe will about double every two years.
• The investment in spending on IT hardware, software, services, telecommunications and staff that could be considered the “infrastructure” of the digital universe and telecommunications will grow by 40% between 2012 and 2020. As a result, the investment per gigabyte during that same period will drop from $2 to 20 cents. Of course, investment in targeted areas like storage management, security, big data and cloud computing will grow considerably faster.
• Between 2012 and 2020, emerging markets’ share of the expanding digital universe will grow from 36% to 62%.
• By 2020, nearly 40% of the information in the digital universe will be “touched” by cloud computing providers—meaning that a byte will be stored or processed in a cloud somewhere in its journey from originator to disposal.