Who’s Using Hadoop? And What are They Using It For?
Editor’s note: This post was updated on April 21, 2017.
So, who is using Hadoop? It’s no secret that Hadoop is massively popular. But just how popular is it? Keep reading for information about Hadoop adoption rates, market share and usage statistics.
If you try to find information about topics like Hadoop market share, you know it can be hard to come by. There is no complete list of businesses that have adopted Hadoop or have immediate plans to do so. Unlike tracking, say, Web server market share – which groups like W3Techs measure by simply crawling public websites – there is no good way to discover how many people are using Hadoop. In most cases Hadoop installations are private.
Sure, you can easily find a lot of speculation from analysts and others about Hadoop adoption, such as, “eighty percent of all Fortune 500 companies will have adopted Hadoop by the year 2020.” Predictions like these sound good, but they are just that – predictions, not hard measures of what is actually happening.
So let’s take a look at actual data regarding who is using Hadoop, which industries have embraced Hadoop most heavily and which challenges companies are using Hadoop to solve. Much of this information is drawn from Syncsort’s eBook, “Hadoop Perspectives for 2017.”
Who Is Using Hadoop? Hadoop Adoption Rates
When Syncsort first began monitoring the Hadoop landscape, Hadoop was a young technology. Circa 2013, studies like this Gartner survey and this one by analyst IDC found that Hadoop adoption rates hovered around 35 percent among all enterprises.
Interestingly, Hadoop adoption rates are still in that range, according to more recent surveys. But that’s not because the Hadoop market is stagnant. On the contrary, according to data from Forrester cited in that same source, the Hadoop market has grown from $435 million dollars in revenue in 2015 to $768 million this year.
So, one key takeaway about the Hadoop market is that although adoption rates have remained steady over the past several years, the value of Hadoop has increased significantly.
How Are Organizations Using Hadoop?
That observation is borne out if you look at how organizations are putting Hadoop to use and what they are now saying about its value.
Syncsort’s findings on Hadoop use with mainframe infrastructure.
71 percent of respondents to Syncsort’s study of Hadoop use said Hadoop is “valuable” or “very valuable” for working with data from legacy systems like mainframes. This suggests that Hadoop is being used not just to address new types of data challenges, but also to improve organizations’ ability to work with the types of data they have already been using for years.
Businesses today are also leveraging Hadoop as a solution for more than just warehousing data. At least half of the organizations surveyed by Syncsort reported that they are using Hadoop to deliver predictive analytics, data discovery, and ETL. It’s clear that Hadoop is becoming part and parcel of their entire business operations, rather than just a place to store data.
Businesses are able to connect legacy data sources on mainframe environments to Hadoop for next-generation analytics and data discovery by taking advantage of data integration solutions like those from Syncsort, which streamline the process of moving data from mainframes into Hadoop for processing.
Hadoop in Finance
Hadoop has proven particularly important in the financial services market, the subject of a special eBook from Syncsort.
As the eBook reports, two-thirds of organizations in the finance industry say that Hadoop helps them increase business agility and increase operational effectiveness.
ETL is also particularly important in the finances industry – even more so than it is among all types of organizations using Hadoop. This makes sense, given that organizations like banks tend to have very large amounts of data that they need to move and analyze quickly using a platform like Hadoop. And in many cases, the data of finance companies still lives natively on mainframes, meaning that a quick data transfer solution like DMX-h is essential for ETL.
Although Hadoop adoption rates have remained steady over the past several years, the importance of Hadoop to business operations has increased. Hadoop is no longer an experimental technology, it is fast becoming a core part of the technology stack at the organizations that are using it.
Read Syncsort’s latest Hadoop survey, Hadoop Perspectives for 2017: New Insights as the Market Matures to uncover the latest trends in data integration!