So You Want to Be a Data Integration Expert…
I recently came across a blog post from Susan Hall over at IT Business Edge on the “Seven Keys to Becoming a Data Integration Expert.” Naturally, the headline caught my attention and I soon learned that it was based on a recent post from David Linthicum on “Obtaining Mad Data Integration Skills.”
As I read through both of these posts, I started thinking. Instead of the order that the seven keys had been originally listed, what if I tried to rank them by how much time and money these things cost organizations during an average month. Here is what I came up with:
- Data governance (this could arguably be number 1, but most organizations I have seen aren’t really doing wholesale data governance)
- Rules and routine
- Database concepts
- Interfaces to data
- Data mediation and transformation
David’s performance criteria states that “…the ability to define how a data integration solution will perform over time. This is very important.” I couldn’t agree more! Building performance and scalability into a DI approach is not only important today, but also for the Big Data requirements of the future. David goes on to say that many DI approaches “become useless after several years.”
We see this every day with our customers and partners. When they’ve hit the wall with their current approach, they often try one or more of the following:
- Add hardware (CPU, memory) – this is expensive and adds to the software cost, and usually does not scale linearly
- Fine tune the approach/tool – this requires very senior IT staff and/or highly-skilled (read: expensive) consultants from the vendor
- Rip out the logic and push it into the database – now you have an ELT approach pushing the cost and complexity into hundreds of lines of SQL and PL/SQL
Syncsort helps customers solve their performance and scalability issues without needing to resort to stop-gap measures that accelerate costs.
Thanks to Susan and David for their posts and the inspiration they provided me to write this one. I look forward to following the discussion on their blogs and reading what they write about next. In the meantime, feel free to leave a comment or challenge me if you want to debate the way I’ve ranked the list above.