Senior Manager, Data Integration
For several years now I’ve been lucky enough to attend and present at MicroStrategy’s biggest user conference, MicroStrategy World, where thousands of professionals from many industries and places around the world meet together for a week to talk about the latest trends in technology, and more specifically, business intelligence. (In the spirit of full disclosure, I’m a former MicroStrategist).
This year, the meeting place was The Wynn in Las Vegas. As always, it was great to catch up with friends, colleagues, customers and partners. Moreover, it was equally exciting to see how business intelligence continues to reinvent itself. This year it looks more energized than ever thanks to advancements in mobile, social, cloud and Hadoop.
I’m sure there will be countless blogs and commentary about conference happenings. Therefore, I want to provide what I hope are different and valuable “takeaways” from the conference ─ a view through the data integration looking glass. So here we go:
1. Mobile and Social Intelligence are key drivers for Big Data. Mobile and social media are creating unprecedented amounts of information. Every Facebook check-in, every like, every comment on social media, provides valuable information about consumer preferences, sentiment, habits, networks, etc. Organizations who can leverage this data will definitely have an edge over the competition.
2. Transforming data is the key to the fourth “V” in Big Data. Transformations – the “T” in ETL – are still one of the most critical challenges organizations face today as they try to leverage Big Data. With increasing volume, velocity and variety of data, what will become even more important is finding ways to capitalize on the elusive “V” ─ value. Similarly, organizations capable of transforming more data in less time, with fewer resources, will be able to answer more “big questions” to provide better products and services to their customers.
3. The elephant in the room is Hadoop. Hadoop has quickly emerged as the framework of choice for Big Data processing and analytics. As such, it is playing a key role in making data processing affordable and disrupting the status quo. During his presentation at “World” Amr Awadallah, CTO and co-founder of Cloudera (a Syncsort partner) talked about how companies are offloading the “T” from expensive proprietary databases to Hadoop. Such a move can shift the economies of scale from as much as $100K/TB to as little as $1K/TB. As organizations implement Hadoop initiatives as a means to scale and reduce costs, they will need technologies to help them unlock Hadoop’s potential.
4. Don’t blame the messenger. In many cases, BI performance and data freshness are a data integration problem. Unfortunately, users often blame the tool that presents the information, in this case, the BI tool. However, more people are starting to realize that behind every successful BI or data warehousing project, there’s a strong ETL foundation. This is especially important when it comes to keeping BI data fresh. Therefore, it is critical to build a high-performance, scalable ETL environment that can seamlessly grow to suit the future needs of your organization.
5. Big Data requires new approaches. During a presentation by Netflix, one attendee asked why Netflix wasn’t using an enterprise data warehouse for Big Data (one that is known to be very scalable but also expensive) The answer from the presenter was simple, but deadly. The so-called data warehouse would never be able to reach the levels of scalability Netflix required. My take? Not all organizations manage petabytes of data as Netflix, or comScore (another Syncsort customer) do. However, they can still benefit from Big Data architectures. As organizations evolve their data processing environments, it’s important to adopt smarter approaches to data integration. A smart approach is one that will scale with the requirements of the business and will deliver results for fewer dollars and with fewer resources. This is exactly what a leading healthcare organization (another Syncsort customer) presented at “World” this year. As they migrated from their legacy ETL tool, they gained faster performance, better standards and best practices, faster deployment times, and enhanced scalability for future growth.
2013 is indeed looking like the year of Big Data, and MicroStrategy World provided more proof of how organizations are quickly embracing the “new normal.”
Well the truth is that some of our customers were fascinated by the enormous potential of Hadoop since the very beginning. However, walking the uncharted path is never easy. Therefore, we worked with many of them to identify the most common barriers for wider Hadoop adoption within the Enterprise. As early adopters and loyal Syncsort fans, our customers also helped us find many areas of opportunity to solve these issues (hint: stay tuned for more).
