ETL 2.0: A New Beginning
Data Integration tools, as we know them today, are failing. Ten years ago, they promised a simple way to load data from multiple, disparate sources; transform it into critical insights and load it into a common repository where business users would leverage it for competitive advantage.
The truth is, most data integration tools can no longer cope with the increasing demands for information. Big Data and mobile computing are obviating the shortcomings of most data integration and ETL tools. A new approach is needed, a new beginning.
That is why today, Syncsort is announcing a new concept that promises to fundamentally change the way people do data integration. We call it ETL 2.0.
ETL 2.0 is about bringing the “T” back to ETL. It’s about realizing the insanity of ELT: a practice that costs millions of dollars a year in database capacity and IT staff productivity (for more details you can read my previous entries Friend or Foe: A Tale of Big Data and Data Integration and ETL vs. ELT: A Tale of Staging and Set Based Processing). We believe data integration tools should be more than just expensive schedulers that push all transformations down to the database. But to do that you need fast, efficient, in-memory engines to deal with the demands of Big Data.
ETL 2.0 is also about bringing the business user closer to their data. During the past 10 years, Business Intelligence tools have done a great job of closing the gap between the user and IT. Reusability, self-service, collaboration, and most recently Mobile BI have all contributed to this effect. Unfortunately, DI still remains largely isolated from the business users; communication is minimal, collaboration almost non-existent. It is time to close that gap and empower the business users by involving them early in the design and throughout the entire development process. Only then, IT organizations will adapt and respond on time to business demands and user satisfaction will increase.
In the end, ETL 2.0 is about achieving strategic business objectives and lowering the total cost of owning and maintaining your data integration environment.
From TDWI World in San Diego, this is our vision for a new beginning. We hope you find it interesting! If you do, please stay tuned for more details…