SILQ

How SILQ Helps Offload to Hadoop

SQL is the lingua franca of the data world. That’s why Syncsort’s SILQ is essential.

Everybody wants to go to heaven, they say, but nobody wants to make the trip. That’s just how IT managers and DBAs feel about moving the data warehouse and other corporate data resources to Hadoop. Being hosted on Hadoop has clear benefits. It’s getting there that is tricky.

One problem? The mountain of SQL code and data spaghetti that make up the typical data warehouse.

It’s easy to criticize SQL for its shortcomings. The science of data management has advanced substantially since SQL first appeared in 1974. The SQL standard has been updated incrementally over time, but the result is a language that DBAs love to hate.

Hadoop
In the data world, the language is SQL.

SQL is essential, however, because it serves as a lingua franca in the data warehouse. Filters and conversion tools ensure that every kind of data, no matter how it was created or how it is formatted, can be manipulated and integrated with SQL.

That means the data maintains its value as operating systems, applications, and data strategies come and go. That’s important.

More important, it means that developers and their work retain their value. Even a modest-sized company is likely to depend upon hundreds of thousands, maybe millions of lines of SQL code. Each data format, each routine made sense at the time, but over the years and decades, the logical architecture of the underlying system can be obscured by ad hoc code, patches, and layers of contributions from different teams and departments.

Hadoop is a cost-effective alternative to big iron for the data warehouse, but some companies have resisted the move. Yes, they reason, the hardware costs less. Data storage costs less. Maintenance is eased and simplified, and Hadoop’s scalability is attractive. But how are we going to keep our code and data alive during the move to a new platform?


SILQ promises to slash the time, complexity, cost, and risk of updating SQL for Hadoop.

That’s where SILQ comes in.

SILQ is an exploration and discovery tool that lets DBAs and IT staff attain understanding and achieve visibility into a large, complex SQL installation.

Industry research shows that it can take several days for a skilled programmer to analyze a SQL routine consisting of just 1,000 lines of code. Deciphering the code, tracing workflows, categorizing tasks and jobs, and updating the code are time-consuming tasks. And they are complex, which means they are subject to error.

SILQ promises to slash the time, complexity, cost, and risk of updating SQL for Hadoop by performing an intelligent analysis of the code and data, resulting in a graphical depiction for the use of programmers, DBAs, and IT staffers.

How much time and money can SILQ save? Analysis of a 1,000-line SQL routine may take a skilled programmer 20 hours. With SILQ, the analysis is performed in less than 30 minutes. That’s a tremendous savings. The resulting flowchart is fully annotated, serving as an aid to migrate SQL workloads to Hadoop as well as documentation for future use.

SILQ brings SQL data to life. Yes, you can get to Hadoop heaven. And with SILQ, you don’t have to dread the trip.

Syncsort’s customers, partners and professional services teams are already using SILQ to easily understand and migrate SQL to Hadoop.  Register here for immediate access to SILQ.

J.D. Hildebrand

Authored by J.D. Hildebrand

Syncsort contributor J.D. Hildebrand dabbled in computer programming and jazz before becoming a magazine writer and editor. He has written hundreds of articles on technical topics for magazines and websites.
0 comments

Leave a Comment

*