What’s Trending in Big Iron to Big Data: The Rise of Mainframe Data and Data Movement Technologies
This blog updates the article, “5 Big Iron to Big Data Trends to Watch” which appeared in Issue 1 (February 2017) of Enterprise Executive Magazine.
The mainframe’s longevity is no longer in question, but its role going forward may be very different. There is industry consensus, supported by numerous surveys and statistics, that the mainframe will continue to play a major role in enterprise infrastructure strategies – particularly in large corporations – for years to come. It’s also clear that the mainframe will continue to be the dependable workhorse for compute-intensive transactions, handling growing data volumes from diverse enterprise sources, including mobile and IoT.
The big question today is how the role of the mainframe is being impacted by evolving IT infrastructures and corporate demands on IT teams. Let’s look at what’s trending in Big Iron to Big Data right now.
The State of the Mainframe Focuses on Big Iron to Big Data
Syncsort recently published its second annual “State of the Mainframe” study, and it provides some interesting insights on trends that are, not surprisingly, top of mind at many of the world’s leading companies today. How do I know? Everyone in my organization, from support to development to sales and marketing, are hearing it from customers this year.
For the survey, Syncsort polled over 250 respondents including data architects, IT managers, developers, business intelligence/data analysts, and data scientists, with 86% coming from organizations with revenues of more than $100 million. The results show a precise picture of how large enterprises plan to use mainframes (and mainframe data) in 2017.
Here’s the interesting part: The hottest trends uncovered by the survey and communicated as top priorities by customers this year reflect the opposite end of the powerful Big Iron compute equation. Mainframes are no longer just about storing and processing a sea of incoming data — though they most certainly still do that better than any other platform. This is evidenced by research from IBM, which found that 71% of the Fortune 500 rely on mainframe data to conduct 30 billion business transactions per day. Additionally there is BMC’s annual study, which finds that digital business has measurable effects on the demand for the mainframe to deliver fast, continuous service.
Another insight comes from Tim Grieser, Program VP for Enterprise System Management Software at International Data Corporation (IDC), who observed that “Mainframes play a key role in digital business as many digital applications are based on mobile or handheld device access to data stored on the mainframe. This is driving growth in mainframe transactions and data volumes.”
So what else are Syncsort’s customers – many of them in banking, healthcare and insurance – focused on in 2017? Increasingly, they are recognizing the tremendous opportunity mainframe data can play as a data source for business analytics. Organizations are powering Big Data initiatives with this key customer and transaction data. It’s all about connecting Big Iron to Big Data.
Let’s look at two of the top 5 trends – both of which relate to the importance of key mainframe data in providing a comprehensive view for competitive advantage with business analytics.
Trend 1 — Mainframe Data Rising as Critical Component in Big Iron to Big Data Analytics
Big Data analytics for operational intelligence, security, and compliance are trending as a critical project in many organizations, particularly large enterprises. The abundance of information collected on z/OS is emerging as prime target to be leveraged for more valuable business insights.
More organizations are looking to exploit this information using analytics tools to provide enhanced visibility beyond what can be gained from traditional on-platform tools. Emerging analytics platforms such as Splunk provide the flexibility to allow data to be used for real-time, visual “dashboard” insights, as opposed to the static nature of display capabilities of existing mainframe tools. In Syncsort’s “State of the Mainframe” study, 60% of respondents indicated they intend to move data off the mainframe for analytics this year.
Many organizations are also investing in tools that can help them access and integrate mainframe data into emerging Big Data platforms like Hadoop, Spark, and Splunk. That’s why our partner Compuware recently announced the availability of Application Audit™, an innovative cybersecurity and compliance solution that dramatically enhances the ability of enterprises to stop insider threats by fully capturing and analyzing start-to-finish mainframe application user behavior. Application Audit’s integration with Syncsort Ironstream enables IT to more quickly discover and take action on security issues and application faults.
Trend 2 — Technologies that Enhance or Monitor Data Movement between Platforms Will Rise in Importance
Data movement across the variety of platforms in distributed enterprises is an important function that must be secured, monitored, and performed efficiently. Files moving onto or off a z/OS mainframe must be tracked to ensure that only authorized parties are initiating those transfers, that the security of the information, whether it be social security numbers, personal health information, or credit information, is protected from unauthorized access, and that the movement of data occurs in a manner which meets the requirements of the business without impacting delivery of IT services.
Big Iron to Big Data Trend: The “State of the Mainframe Survey” showed that more than 60% of large organizations plan to move Big Iron data off-platform for Big Data analytics, with 23% already moving data to Hadoop. Recent visits to customer sites confirm this finding.
In addition, many companies are focused on populating Spark or Hadoop data lakes with enterprise-wide data for analytics, and want to include mainframe data in the mix. Technologies that can enhance and monitor data movement — including what, who, when, and how long — are going to get high priority in budget decisions. Increasingly, organizations are also investing in tools that help ensure data quality going in and out of the data lake, and validating and enriching the information in the data lake to ensure the data lake itself doesn’t become a “data swamp.”
Next week, we’ll look at what else besides connecting Big Iron data to Big Data analytics is trending on mainframe in 2017.