A Closer Look at DMExpress 6.5
Hello from Rat World…I mean Orlando! I am at the TDWI conference here where yesterday, Syncsort made a significant announcement that extends our world-class Data Integration Acceleration to the market. I wanted to take a moment and explain some of the product related enhancements we are adding to DMExpress with this new release, DMExpress 6.5.
For starters, we are adding metadata integration into the DMExpress product. This is one of the most important features we can provide in the world of DI Acceleration. Now we can “snap” in DMExpress into existing DI/ETL environments and co-exist. Many companies have made strategic decisions with DI vendors that have more extensive capabilities than Syncsort (like data profiling, comprehensive metadata management, etc.), but many of you have told us that these products need some performance boost in certain places. Enter Syncsort with DMExpress.
Our metadata integration capabilities will allow our customers to import existing DI/ETL jobs/maps into DMExpress and pre-build DMExpress jobs & tasks. Probably more importantly, we can export back the execution-time metadata into these platforms in order to maintain data lineage. This is very important for compliance, auditing and reporting. Watch this blog for more soon on metadata interchange.
We’ve done a lot more in the DMExpress 6.5 release too. We have added a lookup function. This lookup will be a one-to-one lookup from a reference source file or a database. Because it is implemented as a function rather than a task-level transformation (DMExpress talk), multiple lookups can be performed in one task. One customer I showed this to actually used the word “cool!” Some may say (not me, of course!) that he needs to get out more often, but all joking aside it is going to provide a huge benefit and flexibility to our customers. We can also “push” the lookup into the database if the reference source is a database.
The Teradata Parallel Transporter (TPT) is gaining traction with Teradata customers. Earlier this year, Syncsort added support for the Teradata load and unload utilities, including FastLoad, MultiLoad, TPump, and FastExport. With DMExpress 6.5, we are adding TPT support for the target or load. We’ll add source early next year.
We have also improved developer productivity with this release. First, we’ve added a “diff” capability to see the difference between jobs and tasks, in the context a user can really use. Text AND HTML are supported. The Expression Builder also now maintains formatting, spacing, and indenting for easier reading and maintaining of expression. I’m the first to admit we should have already done this long ago…
Finally, I have to mention performance. Wait, maybe I should have started with this one since we are the fastest engine on the planet! We have improved performance for aggregations with multiple “group-by” keys. We have seen CPU usage decrease by 30 percent, thus also improving overall elapsed time. How cool is that! Okay, maybe I’m the one who needs to get out more often…