Three Big Questions about Big Data: Wrapping up Strata + Hadoop World
Nearly five years ago, Big Data jumped into the center stage. At the time, few people imagined the huge impact this concept would have, not only in the business world but in almost every aspect of our lives. Even back then, the ‘Big Data machine’ was already in motion.
Big Data continues to mature at an enormous pace, with thousands of use cases ranging from national security, to business profitability, to humanitarian causes. As we approach the final stretch of 2014, and with the Strata + Hadoop World Conference in NYC coming to a close, I would like to stop and reflect on some of the most important dimensions of Big Data.
- Technology. There’s a lot to talk about here, and one of the aspects of Big Data that receives most of the attention. So, yes, Big Data is maturing very quickly, and with more mainstream organizations deploying these technologies, we will continue to see discussions and innovation focused on Enterprise-grade capabilities. This includes security, management, monitoring & administration, self-service analytics, automated data profiling & discovery, and transparent integration of legacy technologies with Big Data platforms.
- Law & Ethics. As companies capture every detail of our virtual and physical footprint, the privacy, law and ethics debate is only going to get hotter. As Julia Angwin questioned in the keynote after spending nearly $2,400 trying to achieve “digital” privacy: “Is privacy becoming a luxury good?” People are learning more about Big Data and some of them are increasingly concerned. Who owns Big Data? Can it become (if it isn’t already) a business of trading often-sensitive information? In Julia’s view, if you’re not paying for it, you’re not the customer – you are the product. And more importantly, Can big corporations and governments shape user behavior – our views – and thus the future by leveraging Big Data? According to Shankar Vedantam, the answer is yes. The risk is, he says, more data can make more theories “possible”. This means, people will increasingly use Big Data to reinforce pre-conceived notions.
- Economics. According to John Shitka, SAP, the Hadoop market alone will be worth $50B by 2020! There’s been countless examples of how Big Data can bring significant business value to organizations by improving customer engagement, developing best-selling products and services and so on. However, can we quantify the monetization of these benefits? In essence, how much is data worth? A recent WSJ article, exploring this question noted that “Corporate holdings of data and other ‘intangible assets,’ such as patents, trademarks and copyrights, could be worth more than $8 trillion.” That’s massive by any standards. In some way, the “invisible hand” has figured this out – by adding an important premium to the market value of companies such as Google and Facebook – however, as the article points out, existing accounting regulations are far behind.
It’s important to highlight Big Data should not replace good judgment. After all, as Shankar says “the brain was the first Big Data machine”. Amidst all the change, one thing is certain – Big Data itself – will continue to be a strong transformative force. Thus, it’s in our hands to embrace it. Not only by bringing technology innovation, but also social and economic innovation, defining the rules that govern and shape Big Data.