Much has been said about the increasing speed at which data is generated and consumed – as early as 2011 the McKinsey Global Institute declared that 15 out of 17 sectors in the United States have more data stored per company than the US library of Congress. Needless to say, businesses themselves are changing faster than ever, and in many cases this change is fueled by the same insights this data generates. IT departments not only have to deal with the increasing velocity of Big Data, but are also face unprecedented changes in the way businesses use and consume data.
So, how can businesses and IT catch up with each other? That’s exactly what one of the latest innovations to our ETL solutions is all about. As part of our agile development strategy, in advance of our next major release, we have made available a new user and application interface that makes it very easy for ETL developers and data enthusiasts to develop sophisticated data flows using common words – as close to good’ol English language as it gets. This opens the door for a much broader set of users, not only ETL developers and data scientists but what I would call data enthusiasts. But wait, it gets better. This interface is completely dynamic and programmable, in other words, it functions like an API. Machines can programmatically generate it and parse it. This means you can automatically generate dynamic, business driven transformations to deploy on virtually any platform – from Mainframe to Hadoop and everything in between, including Cloud.
This technology is great for both customers and partners. Some of our customers already have scripts that generate hundreds of data transformations on-the-fly without human intervention. The best part is, by nature these scripts are self-documented and can easily be read by humans as well as by an intuitive graphical user interface. Our partners couldn’t be happier either, as this technology enables them to more easily leverage the fastest data transformation engine to create valuable applications.
Another feature we’ve made available now is enhanced support for JSON and the new web-based console, which allows mobile users to monitor and manage data flows through mobile devices. Just because we’re data enthusiasts doesn’t mean we have to chain ourselves to our desks, right?
Of course, we’re performance geeks, so we’re also packing lots of performance enhancements including the fastest way to load data directly to Vertica from virtually any data source, including Hadoop; parallel output processing to accelerate data preparation by up to 40%; and several other optimizations for joins, sorts, compression, as well as threading and memory consumption.
All these enhancements and more will be part of our upcoming Spring Release. However, with a seemingly endless winter, we decided to get a few pieces out the door early, so you can start warming up and go from cold to cool. Get it on premises or get it on the Cloud… your choice!