Archiving Your Data for Regulatory Compliance with Hadoop and DMX-h
You know that smart use of data can help drive your business and earn you money. But do you realize that poor data management can also cost lots of money in regulatory fees and audit fines? Here’s what you need to do with your data to satisfy regulatory compliance requirements.
The Cost of Poor Compliance
Let’s start this discussion by emphasizing just how much is at stake when it comes to big data and regulatory compliance. It’s not just about pleasing federal authorities sitting in an office so that they don’t write bad reports about it. Responsible data management has a significant and direct impact on your reputation, your relationship with customers and your bottom line.
While the price tag for violation of compliance policies varies on a case-by-case basis, the fines that the government imposes add up to more than pocket change. You don’t want to make the same mistakes as two financial industry companies, for example, who were fined $1.8M and $1.3M in for reporting violations the last two years.
The cost of poor compliance doesn’t end with regulatory fines. Reports of mismanaged data damage your company’s image and undercut your relationship with customers. These factors also cut into your bottom line.
The Challenge of Regulatory Compliance
For most organizations, assuring compliance with regulatory policies that govern how data is stored and accessed is challenging. That’s because the regulations that apply to the use of data, such as the Security and Exchange Commission’s Order Audit Trail System (OATS) policy, present a two-fold challenge. They require that you both store data securely and keep it accessible.
Achieving secure data storage while maintaining accessibility to data is hard. If you only had to do one or the other your life would be much easier.
You could store data in a way or place where no one could ever access it, or you could make it accessible to everyone without storing it securely. But, alas, this is not an either/or choice. You have to achieve both goals in order to stay compliant today.
Using Hadoop to Stay Compliant
Fortunately, platforms like Hadoop can help you meet this challenge. When properly managed, Hadoop provides an environment where you can both secure your data and keep it available to people with the proper privileges to access it.
In this respect, your data stored on Hadoop is much more compliance-friendly than it is if you leave it in a legacy environment, like your mainframe. Mainframes were not designed with modern compliance and access-control requirements in mind, and they are a poor solution for meeting regulatory needs.
Securing your Hadoop instance is beyond the scope of this article, but suffice it to say that there are plenty of resources online to help you doing this. Following the official documentation about running Hadoop in Secure Mode is a good place to start. How-tos like this one take Hadoop security a step further.
Moving Data into Hadoop
The only other challenge you have to solve in order to take advantage of Hadoop to meet regulatory needs is to transfer your data into Hadoop. If you try to do this by hand, you’ll likely find it difficult. Mainframe data exists in diverse forms, is often stored on legacy hardware like tape drives, and can’t be accessed by Hadoop directly.
If you take advantage of tools like DMX-h in order to ingest your data automatically into Hadoop, you can avoid obstacles. DMX-h handles the tedious part of Hadoop ingest for you, allowing you to move data seamlessly from legacy environments to Hadoop.
If you think compliance requirements are just too complicated or difficult to meet, think again. With Hadoop and DMX-h, you can be compliant – meaning you can protect your bottom line and reputation while achieving better data analytics results at the same time.
Legacy data in Hadoop causing unwanted roadblocks? Don’t miss opportunities to maximize the breadth of your data lake – Download our latest eBook, Bringing Big Data to Life, to learn trending insights on integrating mainframe data into Hadoop.