Architecting Mainframe Apps to Siphon Audit Data into Splunk
What makes one application a well-behaved member of the family and another application a mischievous black box?
One answer is audit. An application that does a better job at audit is a more valuable software citizen. It’s also capable of emitting multiple data streams which can be ingested by a utility like Ironstream siphoned into an analytical resource like Splunk.
Without adequate audit data streams, enterprises may no longer have a clear picture of roles, access and data distribution patterns. Speaking of this situation, David Hodgson, the new GM for mainframe business at Syncsort was blunt.
“It is a piece of dirty laundry,” Hodgson told SearchTarget.com in 2015. The focus may be on “using APIs to deliver agile services,” but these strategies cannot fully deliver on promises made by suppliers of expensive technologies without adequate app data for measurement and audit.
Hodgson also told SearchTarget.com that, despite the mainframe’s reputation for hardened security, many organizations are paying closer attention to the security of their mainframe data, as stories of data hacks continue to make headlines. A recent Syncsort study also shed light on why mainframe vendors cannot be complacent about mainframe security.
But many enterprises lose track of user permissions and who has access to what, which poses a possible insider threat to data security. IT pros may have usernames and data descriptions, but not a complete picture of data on the mainframe. New tools are addressing this concern. For instance, Medical Mutual is using Syncsort’s Ironstream to see previously hard-to-access mainframe data, alongside other security information it was already analyzing in Splunk Enterprise. So how can application developers design applications that integrate mainframe audit and logging information that can be made available to Splunk via Ironstream so Splunk can provide operational intelligence and forensics? Let’s look at some reasons for architecting applications in this manner and best practices for providing standardized audit information.
Adding App Audit
Because they are designed to solve problems encountered in countless domains, applications exhibit tremendous diversity. This intrinsic diversity challenges any attempt to develop uniform guidelines as to what ought to be audited, when, and for whom. Yet there are strong reasons to try.
Here are five:
1. Security In order to identify anomalous application behavior associated with malicious activity, indirect measures such as disk accesses, DB2 reads, and memory utilization are often not sufficient.
2. Forensics As the government standards organization NIST notes, audit records can help with reconstruction of events: “Audit trails can also be used to reconstruct events after a problem has occurred. Damage can be more easily assessed by reviewing audit trails of system activity to pinpoint how, when, and why normal operations ceased.”
3. Data Provenance Sometimes it is necessary to trace data back to its source through processing steps. For example, architect Ray Kahn described a data modeling project which involved data profiles for 250 million US consumers, but would be accountable for any leaks of personal financial information.
4. Rollback / Resilience for Composable Services The service composability design principle was developed during the heyday of Service Oriented Architectures (SOA). An application which provided such a service must be audited to verify that it can perform rollback as promised.
5. Compliance Some applications must be verified by external parties. “The audit process must establish if the development processes are in compliance with the rules and regulations imposed by external entities,”assert Cotfas et. al. in their analysis of flexible distributed applications.
Tips and Standards
OWASP tries to make it easy for Web Applications with their Application Security Architecture Cheat Sheet. The standard’s approach extends beyond web applications. For instance, audit streams can log additional details for user session start/end, or trace the click-for-click pattern in which users interact with applications.
Policies are abstractions that map to application capabilities in potentially complex ways. Policies are normally framed in terms of how users interact with an application, but an application interacts with a z/OS ecosystem of services. Audit streams can be enhanced to track policy settings and enforcement.
App Design Revisited, Re-Engineered
Applications design is part craft, part good listening to the requirements set forth by domain experts. Done well, it will provide intelligence for performance, profit, and protection that can extend beyond original design objectives. While the sought-after sweet spot may be operational intelligence, when applications produce additional domain-specific logging, the results go beyond operations management.
Achieving these more nuanced goals is possible in the era of big data. Applications designed in an era when disk accesses and memory use were far more precious commodities can be re-engineered to audit themselves, their users, and their interactions with other system services. As IBM’s move toward cognitive computing extends into more of its operations, these repositories may take on added value. Application audit streams – subjected to AI or machine learning analysis — can disclose patterns in customer and supply chain activities that would not otherwise be possible.
Whether security considerations will overwhelm other uses worries Matt Davies, Head of Marketing EMEA at Splunk. Davies told CBR that “For real-time IoT security intelligence – there will be a need to monitor many kinds of data (often machine data) such as the network, device, applications, infrastructure and user behavior. Industry bodies need to provide real-time security threat…” Perhaps, but a rich applications audit stream is a factor in mitigating risk.
Perhaps, but a rich applications audit stream is a factor in mitigating risk.
App-Ironstream-Splunk as Design Pattern
Putting an audit-instrumented application at the front of a pipeline that moves through Ironstream to Splunk is itself a design pattern.