Big data and Hadoop have had a somewhat schizophrenic relationship with the finance industry. On the one hand, these technologies hold endless promise when it comes to risk analysis, building customer relationships, cross-selling financial products, and even assessing which customers to go after and which job candidates are the potential winners.
On the other hand, finance is notoriously slow to adopt new technologies — as caution and wisdom are the ruling principles of the industry, and big data and Hadoop have not exactly proven their worth until very recently. Machine learning, for example, which is one of the driving forces of the big data movement, does a pretty good job of predicting overall future market trends, but isn’t a good indicator of exactly when to invest in or dump stocks in order to maximize profits. That makes it less than helpful to most financial firms.
However, recent years have brought tremendous development in Hadoop, in terms of security, flexibility, and ease of use. Meanwhile, big data is actually delivering an ROI in other sectors, resolving a lot of the questions about whether it can be a viable tool in finance. The answer is finally a resounding ‘yes’, and jobs are opening up with firms ready to jump on board. Here are the top big data and Hadoop positions in finance right now.
1. Chief Financial Officer
Ah, you say. A CFO isn’t a data position. Currently, it isn’t, but slap a data-related degree on top of that MBA and you’ll see the job offers start flying in. While the CFO has long been relegated to crunching numbers in a back office, data is putting these execs in the limelight, because a CFO with data analytics skills has the potential to be a powerful instrument in delivering operational and business intelligence.
2. Data Scientist
Currently ranked the Sexiest Job in the 21st Century, a data scientist can literally pick what industry they want to go into because, after all, there are job openings everywhere. But if money is your game, there’s lots of it to get your hands on in the finance sector (you make your boss ultra rich, and in turn, they make you pretty rich). Data scientists are professionals who do the sexiest part of data analytics — determining excellent questions to ask the data, convincing the data to spit out those answers, and then packing the answers up in a nifty-cool presentation called data visualization. All very sexy stuff.
3. Data Architect
A data architect isn’t the one who does all the in-depth analysis, but a data scientist can’t even begin to do his/her thing until the data architect struts his/her stuff. The data architect is the one who builds the infrastructure used to collect, store, secure, process, and analyze the data. It involves constructing databases, reformatting data, and building security processes and protocol into the system so that nobody who isn’t supposed to get your data can get your data. It’s kind of like playing Legos all day, only with data and software instead of toys.
4. Data Analyst
There is a lot of discussion in the fields of data science and business about the exact differences between a ‘data scientist’ and a ‘data analyst’. Some employers, indeed, use the terms interchangeably, mostly out of ignorance, but occasionally because they want the more glamorous ‘scientist’ or because they want the more affordable ‘analyst’. Generally speaking, the analyst is more involved in the SQL programming and the mid-level statistical analysis, while data scientists are more involved with data manipulation and advanced analytics.
Basically, if you’re qualified for these positions and are looking for a job, depend more on the job description than the title to guide you to find the best fit for your skills and comfort level. Don’t forget to look for the potential for upward mobility, as well as the opportunity to work with experienced data scientists whom you can learn from and network with for career advancement.
5. Software/Data Engineer with an Expertise in Data Science
When it comes to making big data and Hadoop dance a jig, that’s left up to the software engineer with knowledge in and experience with data. While the pure software engineer focuses their efforts on building software applications, software engineers/data engineers spend much of their time working with queries and building Hadoop and/or Spark services to meet the needs of the firm.
The good news is, any candidate with a Computer Science degree can get their foot in the door as a data engineer with incredible potential for earning and ladder-climbing. Although, the potential for the pure software engineer isn’t shabby, either. It’s just that data is such a hot commodity, there will be jobs available to the candidate with ‘data’ in their skills set that might not be open to the simple, old-fashioned ‘software engineer’.
Syncsort has Big Data solutions that are helping these professionals meet data integration, migration and transformation challenges necessary to leverage big data in the finance industry. Important to the larger finance industry companies, their software also seamlessly accesses and integrates Big Data from the mainframe into Apache Hadoop and Apache Spark. It also allows organizations to work with mainframe data in Hadoop or Spark in it’s native format, which is essential for maintaining data lineage and compliance.