Coming Soon to a Mainframe Near You: Machine Learning, Part 2
In Part 1, I discussed some compelling new uses of mainframe machine data that go far beyond today’s common use cases. I looked at how Google is using interaction between machine learning teams and the rest of its employees to drive transformative thinking in all product development, and began to explore how ML is part of predictive analytics on z/OS. Now, let’s look at what IBM, Elite Analytics, Syncsort and Splunk are doing to leverage ML for next gen analytics, and how bots can become the development teachers of the very near future.
IBM is already using Predictive Failure Analysis inside z/OS to anticipate certain types of failure — based on ML using data on its own behavior. Predictive Failure Analysis (PFA) pulls data from IBM Health Checker for z/OS and uses ML to recognize opportunities to alert operators to potential problems in advance of a serious problem. PFA events include storage usage, queue growth rates, log record arrival rates, message arrival rates, private storage exhaustion, and SMF arrival rate.
►It’s easy to see how third party tools such as Syncsort’s Ironstream® could increase the variety and volume of data sources for z/OS ML. Ironstream already helps organizations that use Splunk Enterprise derive operational intelligence from the vast amounts of machine data generated by a mainframe, and Syncsort continues to add new z/OS data types to the list it can deliver in real-time to Splunk. Currently Ironstream collects z/OS log data, including over 40 SMF record types, Syslog, SyslogD, log4j, Unix System Services file data, DB2 tables, and more from mainframe systems and forwards it securely to Splunk Enterprise, Splunk Enterprise Security, and Splunk Cloud platforms. While the design pattern tends to pair Ironstream with Splunk, there are other pairings in which ML could be unleashed on mainframe data.
Elite Analytics bills itself as an expert in predictive analytics for SPSS. Elite believes that support for Spark on z/OS will further give “z/OS shops real incentive to explore implementing real-time predictive solutions for their mission-critical applications without having to move data off-platform.”
Flavors of Prediction Algorithms
Back in 2013, firms such as Swift IQ forecast that demand would emerge for Machine Learning as a Service — a sort of Salesforce.com for ML. In an interview on Machine Learning as a Service, Swift IQ’s CEO Jason Lobel suggested four types of use-case categories where ML can shine:
- Recommendation engines; e.g., Netflix, eBay, Amazon shopping suggestions
- Frequent pattern mining, such as grocery store shopping patterns
- Classification, such as organizing keyword search results based on searcher relevance
- Clustering, which can be used to identify high-value customers
So it was just a matter of time before Amazon Web Services announced its MLaaS offering, featuring “popular ML use cases” drawn from these categories.
There are few mainframe legacy apps that would not benefit from insights gained through one of these ML algorithms.
Advent of the Developer-Teacher
The kerfuffle over Microsoft’s Tay, the bot that got its human handlers in hot water for its uncouth racist message, points to the importance of good data in ML. So-called “unsupervised ML” (the parent-child analog is intentional) is appropriate for some settings, such as geospatial analytics or segmentation. Supervised learning involves construction of decision trees based on historical data, which are then used to predict future outcomes, or to insert classification values into existing records.
Whichever method is selected for a given problem, a case can be made that developers armed with ML tools will become tomorrow’s developer-teachers.
Players familiar with Doom, the first person shooter game, would likely not be surprised to learn that ML is the subject of a contest sponsored by the IEEE Computational Intelligence and Games conference. ML researchers are tasked with creating bots that can play the game based only on visual input — just as a human would.
Once that ML assignment has been checked off, the concept of a person interacting with a fully smart device like Tom Cruises’ gesture-based UI interaction in Minority Report, won’t seem far-fetched at all.
Skynet? Maybe, but it will take more than ML.