“Survey Says . . .” The phrase is from a popular culture show Family Feud, which debuted back in 1976 and has aired in one form or another ever since. In 2013. TV Guide placed it in third place in a list of the 50 greatest game shows of all time.
The popularity – and certainly the importance – of surveys goes far beyond this lightweight reference. While organizations like Gallup and the U.S. Census are recognized by most Americans, hosted survey tools like SurveyMonkey have vastly increased the sheer number of citizens surveyed. In 2013, SurveyMonkey reported that it had as many as 15 million registered “users.” Using the language of survey specialists, the sample sizes and the sheer number of measurements taken has increased vastly.
Credit: Rich Stowey via Flickr
Less widely known but highly respected is the National Opinion Research Center (NORC) at the University of Chicago, formed in 1941. NORC is responsible for many significant surveys, including the National Immunization Survey, the National Flu Survey, Survey of Consumer Finance and the General Social Survey. The latter survey, sponsored by the National Science Foundation, is the second most analyzed survey dataset – after the US census.
Samples, Soundness & Sophistication
Survey analysis has required computational resources since the 60’s. One of the longest-lived software in history is SPSS – originally the Statistical Package for the Social Sciences, since 2009 owned by IBM. Widely used as a general purpose analytics tool, in the early days it was often employed to study survey data. SPSS and analytical resources like it are now employed to tabulate, analyze and predict outcomes based upon survey data stored in data warehouses – where Syncsort Big Data ETL may well have been employed to load and standardize survey data to make it amenable for analysis.
Traditional paper-and-pencil, clipboard-ready surveys had intrinsic limitations. The cost to administer each survey could be nontrivial. Scoring the results introduced delays. The types of questions – “stimuli” in a cognitive psychology sense – were relatively limited. Questionnaires required time-consuming prototyping and test phases, which likely limited the quality of many surveys.
Rise of the Techno-Survey
While demand for survey research contributed to the history of computing, it is less widely understood that computing has likewise changed the face of surveys. There are a number of factors that have led to the rise of the Techno-Survey:
- Increasing availability of supplementary demographic and other variables through Big Data variety and volume
- Computer administered surveys, useful in significant settings, have lower cost per survey
- Survey development allows for greater testing and refinement prior to launch
- Mobile devices have expanded ease of use and access
- Adaptive surveys with demography- and response-conditional presentation of questions
- More sophisticated “questions,” including video, immersive media, virtual reality
- Auxiliary response measurement, such as reaction time, eye movement
- Coordinated collaborative surveys, reflecting group opinion or decision processes
- Ability to overcome accessibility limitations for special populations
- New impetus for re-analysis of legacy survey data by leveraging Big Data variety
- Integration with conventional and adaptive testing within eLearning applications to refine just-in-time learning objectives
Surveys and Democracy
Political surveys border on the very annoying during election season, but the effective use of survey data made a name for the likes of Nate Silver. Named by Time as one of the world’s 100 most influential people, Silver has written not only about his successes but about the limitations of Big Data. Silver cites missing data, chaotic systems and feedback loops as instances where Big Data will not help with prediction. In his book The Signal and the Noise, he worried that Big Data is flooding data warehouses with “noise.” He cautioned that Big Data does obviate the need for careful experimentation in research – and that includes survey research.
Surveys have a role to play in democracy. In periods between election cycles, surveys can take the pulse of the people and guide public policy. Examples of policy-related findings abound, but one of most discussed is attitude toward gay marriage. According to the Pew Research Center, “the rise in support for same-sex marriage over the past decade is among the largest changes in opinion on any policy issue over this time period.”
Can surveys tell us how this came about? Despite Silver’s apt reservations, perhaps Big Data and the Techno-survey could help answer this and questions like it.
Still, even in a democracy, surveys have public policy limits. More than a hundred years ago, De Tocqueville wrote:
“In America the majority raises formidable barriers around the liberty of opinion; within these barriers an author may write what he pleases, but woe to him if he goes beyond them.”