Enter the Big Data World: Key Takeaways from Strata + Hadoop World
Another invigorating Strata + Hadoop World conference came to an end on Friday. Aside from all the excitement around Big Data technologies, vendors and projects, it was nice to catch up with really good friends, colleagues and partners.
As we all go back to our daily routines, here’s my usual list of takeaways, and this time I would like to reflect on three key themes at San Jose.
- Real-time Technology. Algorithms to answer the most relevant questions, frameworks to process more data in less time, interfaces to allow users to better interact with their data, hardware to run faster and more efficiently – all of these dominated the conference. However, this time the emphasis was on real-time. As Hadoop moves away from the batch oriented MapReduce framework into more flexible and efficient frameworks such as Spark and Tez, real-time collection, processing and distribution of data is quickly becoming the norm. In the words of Anil Gadre from MapR: “real-time data to action is what really matters.”
- Privacy & Security. As Hadoop adoption goes mainstream, with more and more data stored and processed in the open source framework, security has taken center-stage. In this context, it’s no surprise that BlueTalon was recognized at the startup showcase for its efforts to provide cell-level security for Big Data; or that Cloudera’s Eddie Garcia opened Friday’s keynote asking the community to build secure by default. However, there’s another aspect that, I believe, we’re yet to answer: Privacy. Although a recurring theme at Strata, there’s still lots to be done to create even more awareness and education around this critical topic. As mentioned by Solomon Hsiang, data is playing a much more central role in the way we govern ourselves. Yet, according to a presentation by Alysa and Lauri from Kelley Drye, 90% of US consumers feel they have lost control of how companies use their personal data! I expect security and privacy to get even more attention within the next 12-18 months.
- Social Impact. As expected, we were reminded again about the huge impact that Big Data technologies are having on every aspect of our lives. I’ve discussed this several times before. But, even beyond the dramatic changes we’ve all seen – social media, mobile technologies, wearables – we’re just scratching the surface. Alistair Croll warns us “when machines get more intelligent, we may not notice because they may be us & we may be them.” The truth is, even the best algorithms cannot predict what the future holds for us. Besides, we have creativity on our side, and we also have the past. In words of Eden Medina, “We have a lot to learn from our past, and that past is global.”
I guess a list of highlights from Strata would not be complete without mentioning the surprise appearance of DJ Patil, US first Chief Data Scientist, who reminds us that data is everywhere; every company, every person, and now even governments are quickly becoming data driven.
Syncsort’s booth at Strata San Jose 2015
Also of interest, Tendu Yogurtcu, GM for our Big Data business, had a great interview with Wikibon’s Principal Researcher Jeff Kelly and SilconANGLE Founder John Furrier on theCUBE, broadcasted live from their BigDataSV event during Strata+Hadoop World. Tendu discussed Syncsort’s strategy to grow our Big Data business. She stressed that we continue to invest on several fronts, evidenced by our newly announced acquisition of William Data Systems, contributions to the Apache Hadoop open source project, innovations in our new DMX-h 8 release and partnering on the BigFrame initiative with Cognizant and Cloudera for Mainframe offload. She also discussed how the William Data acquisition is very complimentary to our strategy of making all data accessible. You can view the interview below.
That’s all for now! Happy travels and see you all at Strata New York!