Is Big Data dead? Short answer: No. It’s alive and well – and very valuable, as long as you take the right approach to data management and analytics.
The question of “Is Big Data dead?” is not new. People have been proclaiming its death since at least 2013. The concept of Big Data has been applied too broadly to have meaning, they say.
Others claim big data tools are too specific and customized to build a real ecosystem. And then there is the idea that data just can’t be used to predict things as accurately as Big Data fans have contended.
That last notion may seem especially true in the wake of events like the 2016 U.S. presidential election. Without getting partisan about the issue, suffice it to say that the vast majority of predictive models, which were supposedly based on polls and other predictive data, totally missed the mark when they suggested that the candidate who turned out to be unsuccessful had a very high probability of winning.
Is Big Data Dead? No, It’s Alive and Well (and Living on a Server Near You)
Some of these criticisms of Big Data and data analytics are fair. Can Big Data solve all of the world’s problems and get things right 100 percent of the time?
Of course not. But what can?
The fact that Big Data is imperfect – that predictive analytics sometimes turn out to be wrong, that not every Big Data product out there succeeds in the market, that the term Big Data is subject to the same conceptual ambiguity as other tech buzzwords, like cloud – does not mean Big Data is dead.
Doing Big Data Right
It just means that, like any other mainstream technology, Big Data has to be implemented in the right way in order to be effective.
If you do Big Data the wrong way, you won’t get the results you’re hoping for. It will be easy for you to draw the conclusion that Big Data is dead and just doesn’t work.
But, if you do Big Data the right way, you’ll realize just how much insight it can deliver. Doing Big Data the right way includes adhering to principles like these:
- Standardize your data.
In order to work with Big Data effectively, you need to get all of your data into a standard format – which is tough when the sources of your data are diverse. If you fail to standardize, you’ll get inconsistent analytics results because of poor data quality.
- Use the tools of your choice.
You should empower yourself to use whichever Big Data tool is the best fit for your needs. If you want to use Hadoop, use Hadoop. If you like Spark, adopt Spark. This flexibility of choice is only possible when you have data integration solutions in place that can work with your data no matter which format it is in, or where it lives.
- Make your data really, truly big.
The bigger your data – meaning the more information you collect – the more accurate your analytics results will be. You should, therefore collect as much data as you can – which is possible as long as you collect data from all of the systems in your infrastructure, and put all of your resources to work in storing and processing data.
- Analyze in real time.
Analytics based on historical data are good, but real-time analytics are much better. They tell you what the situation is right now, not what it was a minute or an hour or a day ago. For this reason, you want to be able to integrate and analyze all of your data in real time, without barriers (like incompatible data formats or data extraction delays) that turn real-time data into historical data.
Syncsort’s data integration solutions can help you achieve all of the above. They empower you to collect all the data that your infrastructure generates – whether it originates from mainframes, commodity servers or anywhere else – to analyze that data using whichever Big Data tools you want to use and to do it all in real time.
Big Data’s not dead. It’s just dead to those who approach it from the wrong angle.