Expert Interview with Thomas Powell of ZingChart on Big Data visualization
It’s the age of big data. And while that gold mine of information might be getting a lot of buzz because of its potential to grow business and spur innovation, it can often turn into a big mess.
Thomas Powell, founder and CEO of web-based charting solution ZingChart, says that big data often needs to be cleaned and purified before any meaning can be distilled from it.
“Today it is easier to collect more data than we may know what to do with,” he adds. “Of course, data is often messy and rarely do people want to spend time cleaning it up.”
In addition, while tools might allow us to consume large sets of data more easily, these tools can’t actually think for us and can be easily abused. “We have seen much of the desire of trying to simply ‘plot and know,'” Thomas says. “The more meaningful approach would be to explore and refine data before leaping to apply the correct visualization and conclusion.”
We recently checked in with Thomas to learn more about how ZingChart has evolved in the age of Big Data, get tips for business on managing data and find out about trends in data presentation. Here’s what he had to say:
Tell us about ZingChart. When and why did you create this software?
Who should be using ZingChart?
ZingChart is a solution for developers who have figured out that most open source solutions are quite lacking in feature sets and don’t scale well with large data. Our sweet spot is clearly not a simple pie chart or bar chart. But if you have complex code, are custom code driven, or use big data focused visualizations or dashboards, ZingChart is a good fit.
Why are charting programs like yours becoming more important to business today?
Viz libraries like ZingChart allow developers to build bespoke dashboard and datadriven solutions much faster than in the past. Bespoke visualizations are useful for folks who are not interested in a general solution that may include many extraneous features or have steep learning curves.
In the age of Big Data, how has ZingChart evolved to capture users’ growing interest in data analysis?
Well, the first thing we have focused on is actually visualizing big data. We noticed that many libraries are not effective, even at 50,000 data points. We can work on data sets many times larger than that with interactive features intact. And our experimental z9 library can plot many millions of data points quite quickly. We feel that big data requires reasonable outcomes approaching real-time speed and rich APIs so that the data can be interacted with for filtering and analysis. We dub this approach to the visualization of larger data sets in an interactive manner “BigViz.”
What do you think businesses can do to make the process of sorting all the data they gather easier?
The simple answer is probably not technical, but organizational and somewhat philosophical. Attitude-wise, we think businesses should not conflate big data with big wisdom. Gathering a bunch of data doesn’t make it obvious what it means.
Second, we think that data-focused organizations should do their best to install a skeptical view of their data. We have seen many instances of confirmation biases and quick conclusions drawn from data. Data and decisions made with it are messy and sometimes unclear. While relying on data is better than guessing, data-driven organizations really need to maintain a healthy degree of skepticism.
What are the biggest frustrations or complaints your clients come to you with?
The main situation we run into is the demo illusion problem. Lots of big data and visualization tools have very impressive demos. However, when these tools are applied to real-world tests, they suffer greatly or require a tremendous amount of integration work. Quite often, viz solutions are much harder to use or less feature-rich than the users had come to believe. It seems somewhat ironic to us just how often data-driven technologies are selected with little information other than:
- A solution appears high in a Google query
- A tool is widely used as opposed to really testing a tool to see if it meets project criteria.
What trends are you noticing in infographics today?
We continue to see an emphasis on USA Today-style infographics and various “eye candy” style charts. Historically, such “data bite” presentations with cartoonish graphics were somewhat derided. Nowadays, we see them creeping into business intelligence and IT dashboards. While some may call this trend “chart junk,” it is hard to tell if that is the case; humans want their content consumed by the users. One person’s data junk is a useful report for another person.
Why do you think businesses and audiences love them so much?
Most people seem to consume visual information easily. When well designed, infographics allow complex stories to be told quickly and be consumed easily by their viewers. Of course, building such infographics can be difficult, so it isn’t guaranteed that an infographic will succeed better than a potentially boring report even if it is pleasing to the viewer.
What are some of the most exciting innovations or trends you’re following in your field?
Clearly, mobile reporting is quite exciting since it moves data consumption and collection outside the enterprise and into the field. Obviously, data use on mobile devices is a bit different than traditional desktop business intelligence. There is quite a bit of innovation still to be done. However, the constraints that mobile presents seemed to have started to inspire more creative data solutions than the traditional print reports put into pixels that have dominated much of the BI space for the last 20 years.
We don’t have any one particular trend that we see as the most exciting, but we are impressed with the kinds of solutions we see being developed with our library and others. We like to think that we are like “data concrete.” Interesting things can be built with concrete, though when you look at the raw material, it isn’t obvious what will eventually emerge.