Visualization for Big Data

If a picture is worth a thousand words, is a picture that’s a thousand times bigger better still?

These and other questions are being asked by user experience designers as the tidal wave that is Big Data rolls into the human-computer interaction space.

In recent years, several IPOs reflecting increased demand for business intelligence software demonstrated the need to put data management in the hands of analysts, not only database admins. These IPOs included Qlik Technologies (2010, current market cap $2.4B), Splunk (2012, $8.2B) and Tableau (2013, $4.7B). Earlier generations of BI software provided visualization tools, but Tableau, QlikView and Splunk were part of a crop promising to extend business visualization to a wider audience with fewer intervening steps.

ETL provider Syncsort, Cloudera and Tableau discuss these and other issues in a March 2014 presentation

Data as the New “Soil”

David McCandless asserts in his 2010 TED talk that “Data is the new soil,” and that visualizations are flowers derived from plants tilled in this ground. He believes some analysts – even untrained ones – have a sort of “dormant design literacy” about visualizations. He says that by ingesting a steady diet of web visualizations, users have unintentionally developed expectations about what they see and how they want to see it. In the dense information jungle, he argues, visualization is a relief, “a clearing.”

Marching a bit out of step with Big Data has been Big Software, which has also seen a rise of visualization tools. Software by Irise, reportedly adopted somewhere within 500 of the Fortune 1000, relies on visualization to improve efficiency and transparency in the software development life cycle. Part of what makes Irise attractive to Big Data developers are its gateways to fast-growing repositories like HP Quality Center and IBM Rational.

Also growing from this “soil” are visualizations that bridge data and knowledge. Think Visual Studio on steroids, capable of navigating both data and semantic space. What’s really needed, wrote Bret Victor in 2011, is a systematic approach to interactive visualization, a capability to navigate “up and down the ladder of abstraction.”

Download our eBook: The New Rules for Your Data Landscape

Look Up! It’s Cloud Viz.

Most of the familiar names in BI have roots in client/server architectures, though most offer some sort of cloud version by now. Products like Birst Visualizer, ChartIO, RoamBI and ClearStory are newer cloud-based offerings. BigML, aimed at DIY predictive analytics from the cloud, includes a visualization component. For instance, Ayasdi uses “topological data analysis” to “highlight the underlying geometric shapes of your data and allowing for real-time interaction to produce immediate insights.” Microsoft’s Power BI offers data visualization capabilities supporting million-row tables for Office365 users.

Big Data Visualization

Microsoft Power BI Demonstration (Screenshot)

“Show Me” (Don’t Tell Me)

Think your data challenge is big? Please, to borrow from the Rush song of the same name, show me, but don’t tell me. Big Data visualization is about saying more with less.

Research is underway to explore new ways to visualize big data. Here is one example.

A report by Aditi Majumder and Behzad Sajadi demonstrates how multi-projector displays can deliver visualizations for uses such as immersive simulation or scientific data visualization. They have augmented their displays with gestural interfaces that allow one or more users to interact with these display in real time, performing tasks such as navigating 3D models. In addition, the research demonstrated a capability to light any object to act as a screen for visualizing data – using them as “the visualization medium,” even for everyday objects.

Show Me a Story

There’s more to today’s visualizations than flipboards and hi-res maps. In Business Storytelling for Dummies (Wiley 2014), Karen Dietz and Lori Silverman tell us that visualizations don’t have to tell a story. But, they say, “if you want to use data and data visualization to move people to action, then melding story and data together is essential.” They offer prospective storytellers a few guidelines:

  • Show, don’t tell. Try removing text to see if the visuals still work.
  • Be clear about what you want to accomplish. Dietz and Silverman recommend revisiting the knowledge hierarchy associated with the concepts.
  • Be wary of eye candy. A pleasant-looking visualization doesn’t guarantee that viewers will process the intended message.
  • Slickness and flashiness aren’t necessary. Beware of “. . . thinking that the data is what needs emphasis, when the real key to success is the story about the data (p.155).”

Related: Expert Interview with Thomas Powell of ZingChart on Big Data Visualization

Visualization for Four V’s

Some aspects of good visualization will remain unchanged from the pre-Big Data buzzword era. So what is new? What will need to change by virtue of increased volume, velocity, variety and veracity?

The impact of Big Data will be felt in unanticipated realms. A recent New York Times story identified Big Data genealogy as leading to potential insights into disease inheritability and untold family connections to major historical events. Author A.J. Jacobs explained:

Last year, Yaniv Erlich, a fellow at the Whitehead Institute at M.I.T., presented preliminary results of his project FamiLinx, which uses Big Data from Geni’s tree to track the distribution of traits. His work has yielded a fascinating picture of human migration.

Even a modest homegrown family tree can be difficult to print. Splicing together multiple pages is the norm for DIY genealogy. A fully interlinked tree that connects to the trees created by hundreds or thousands of others demands a scalability that earlier UX design simply lacked.

Beyond the Visionary

Big Data may not lead to bigger stories. Analysts may discover that some old methods – bar charts, line graphs, heat maps – work as well as ever. But a big dose of any one of the V’s – especially velocity – can change all that. Big Data visualization will have to change accordingly.

But in many other domains, animation and 3D may need to become as commonplace as bar charts.

Big data analytics and visualization has shifted the entire data landscape from IT to the Business. Discover how this new data supply chain impacts how data is moved, manipulated, and cleansed – download our new eBook The New Rules for Your Data Landscape today!

Mark Underwood

Authored by Mark Underwood

Syncsort contributor Mark Underwood writes about knowledge engineering, Big Data security and privacy.
1 comment
  1. […] to an article entitled “Visualization for Big Data”, David McCandless said that data is like new soil and visualizations are the flowers of the plants […]

Leave a Comment

*