In 2017, The Economist magazine reported that oil was no longer the world’s most valuable resource, naming data as the oil of the digital era.[i] Data is certainly flowing freely, with 2.5 quintillion bytes created each day.[ii] Like oil, the raw product can be transformed into something useful through refinement – and that is where the market research industry excels. By leveraging the development, trust, engagement and insight it has built over many years, the industry will continue to shine despite the privacy and security uncertainties of the digital age.
Crunching the numbers
Digital data analysis for business is nothing new. It has been around since the 1990s, courtesy of basic number-crunching software. While it wasn’t until 2005 that the term “big data” was coined, the scope and variety of data collection and analysis of information has long shaped decision making and investment across multiple industry sectors.[iii]
Human and machine-generated data is expected to increase exponentially as the Internet of Things and Artificial Intelligence facilitates seamless data gathering. With this growth comes the question of whether digital data can be transformed into relevant and actionable information.
The GreenBook Research Industry Trends (GRIT) Report for 2017 Q3-Q4 notes: “In the Gartner hype cycle Big Data has definitely reached the trough of disillusionment stage within the market research industry with a wide range of challenges identified surrounding access, use and the analysis of new forms of data.” The GRIT report also states that some in the industry were questioning the real value of new data forms, realising that data in itself “was often of little value”.[iv]
The success and usefulness of big data remains tied to both Quantitative and Qualitative research, as they afford the ability to scale and drill down on big data elements and to test, flesh out variables and gain greater insights. In other words, to give big data deeper meaning.
Value and quality co-exist
Regardless of big data hype, business-to-business and business-to-customer relationships and their underlying principles remain increasingly relevant.
When choosing a market research firm, the 2017 Q3-Q4 GRIT Report notes that data quality and price are amongst the key drivers of selection. The report highlights that 27% of respondents believe that data quality is important, followed by the relationship with the person or organisation, service levels and pricing at 10.3%.[v]
While search engines, CRMs, mobile devices, loyalty cards, social media and more capture data that quantifies behaviour, the data’s value decreases as deeper insights into behaviour are required. As the Data Decisions Group writes, “the obvious Achilles’ heel in the big data silo is the lack of attitudinal data — you can see what customers do (and predict what they will do), but you do not know the ‘why’.”[vi]
Various industries are exploring analytical opportunities using big data and Artificial Intelligence (AI). But, they also recognise the importance of quality market research in decision making. For example, the publication Pharma Voice says, “while slower to adopt AI than the retail and financial industries, the pharma industry is now exploring AI. Experts predict that during 2018, AI will give companies more predictive insights more quickly, and at lower costs. These insights will need to be matched with qualitative solutions to better understand the why behind the data.”[vii]
Big data’s current image problem
Big data is fed via channels that involve the passive collection of personal information. While this form of data gathering does not have the same hurdles as market research panel recruitment and retention, it has its own associated issues that continue to be emphasised. When commenting about the passive collection of data, the Massachusetts Institute of Technology (MIT) states “as the user community becomes more aware of the data collection techniques currently being used on the Internet, fears of misuse of data arise. With increasingly sophisticated data collection techniques, users fear discrimination and invasion of privacy.”[viii]
Data breaches like the recent Facebook Cambridge Analytica scandal, where up to 87 million users may have had their data harvested by Cambridge Analytica’s app, This is Your Digital Life, have put people on edge and created hysteria in the community. The incident certainly hasn’t done big data any favours, with the News Portal Dataconomy noting, “a lack of transparency about big data and a lack of reimbursement for users whose data is sent to developers are two very distinct issues highlighted by this scandal. There was no option to opt out and no freedom to decide where the data went or who gets to look at it.”[ix]
Market research firms should be in a position to capitalise on the systems and processes they have put in place to uphold data privacy, but they must also continue to lead in this space. Stable Research has previously addressed this issue, believing that systems should be put in place that give both suppliers and panel members confidence.[x]
Market Research firms should be promoting the efforts they put into privacy and data security. Stable Research has developed the following privacy and cyber-security procedures that are clearly outlined on our company website:
- Policies that relate to IT, Human Resources, staff recruitment, a cyber security checklist, permissions and security levels.
- Data protection through two-factor authentication, remote access password protection, system backups, virus protection, data encryption, penetration tests.
- Physical protection including alarmed premises, data held in Australia, firewalls, no removable devices.
- Cyber Insurance, Public Liability Insurance, ISO 20252 and 26362, Privacy Policy and AMSRO membership, Disaster Recovery Policy and Business Continuity Plan.
‘New’ traditional research
In the latest GRIT report, Rebecca West, Global VP of Market Research Services, Civicom Market Research Services, called for a new definition of traditional market research services in light of technological development. West explains that ‘for most modern researchers, web-enabled IDIs and focus groups and online communities have now evolved into what is known as ‘traditional’. These are now the ‘new’ traditional, and they are alive and well. Neither data analytics nor big data have yet replaced them. There remains the ultimate objective of market research: understanding the ‘WHY’.”[xi]
Spend v outcomes
At present, 65% of market research customers use traditional services “to answer important questions that cannot be answered with other data sources.”[xii]
Market Research organisations should be reminding customers of this fact through active marketing strategies that highlight benefits alongside those that big data provides. Spending throughout 2017, along with projected revenue growth, gives some indication that regardless of publicity around privacy and security issues, big data will continue on its upward trajectory.
In 2017, IDC projected a 12.4% rise in spending on big data and analytic products and services – with revenues expected to grow above $150 billion. Also, “by 2020, revenues from big data and analytics products and services will eclipse $200 billion”.[xiii]
While the report from IDC did not provide a detailed breakdown of “products and services”, the 2017 Greenbook did show that market research clients spend achieved “moderate growth” throughout the previous year.[xiv]
The value proposition of market research organisations must continue to be clearly articulated to gain the attention of organisation decision makers who may be swayed by the “bright lights” of data collection.
The big data giants using survey to gain greater customer insight
Even with the scale of data available to them for strategic decision making, both Facebook and Google actively engage people to provide more information via online survey.
In 2017, Mario Callegaro, Senior Survey Research Scientist of the Ads and Commerce User experience team at Google London, gave a presentation titled Importance of Surveys in the Era of Big Data. In highlighting the differences between big data and market research, he noted, “Surveys can help understand … attitudes, opinions, why people do certain things big data can help understand … behaviours – what people do”.[xv]
He also provided examples that, regardless of the wealth of data gathered through Facebook, the social media giant still considers surveys relevant in gaining deeper insights into what people are thinking and why they do certain things.[xvi]
Confidence in the future
Big data providers currently have a serious issue with public perceptions about privacy and security. Their ability to implement foolproof security measures will continue to be tested by hackers and those keen to capitalise on this valuable commodity. In contrast, market research firms can take additional measures to ensure client information remains secure, in turn creating greater public buy-in working alongside industry groups and others.
Big data and AI will definitely take businesses into new and exciting directions. The concept of traditional research will also continue to change and interlink with big data to provide people and organisations with an even greater understanding of the how and why. There are definitely exciting times ahead for the market research industry.
[i] The Economist. 2018. The world’s most valuable resource is no longer oil, but data – Regulating the internet giants. [ONLINE] Available at: https://www.economist.com/news/leaders/21721656-data-economy-demands-new-approach-antitrust-rules-worlds-most-valuable-resource.
[ii]IBM. 2018. 10 Key Marketing Trends for 2017. [ONLINE] Available at: https://www-01.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=WRL12345USEN.
[iii] Big Data and Cambridge Analytica: 5 Big Picture Truths – Datamation. 2018. Big Data and Cambridge Analytica: 5 Big Picture Truths – Datamation. [ONLINE] Available at: https://www.datamation.com/big-data/big-data-and-cambridge-analytica-5-big-picture-truths.html.
[iv] GreenBook. 2018. GRIT Sneak Peek: The Top Challenges & Opportunities in Research | GreenBook. [ONLINE] Available at: http://greenbookblog.org/2018/01/08/grit-sneak-peek-the-top-challenges-opportunities-in-research/.
[v] Issuu. 2018. Grit report Q3 – Q4 2017 by GreenBook – issuu, p.55 [ONLINE] Available at: https://issuu.com/researchshare/docs/grit_report_q3_-_q4_2017.
[vi] David W. Wilson. 2018. The New Insights Imperative: Market Research + Big Data + Predictive Analytics. [ONLINE] Available at: https://www.datadecisionsgroup.com/blog/bid/153711/big-data-and-market-research-what-is-it-and-how-can-i-get-started.
[vii] PharmaVoice. (March 2018) [PDF] New Jersey: PharmaLinx LLC, p.34. Available at: https://www.frenovarenalresearch.com/wp-content/uploads/PharmaVOICE0-318_Market-Research_AI_Big-Data_Frenova.pdf
[viii] Matsuda, Rosenstein, Scovitch, and Takamura. 1998. Direct Marketing on the Internet. Data Collection: Defining the Customer, (1) p2.
[ix] Dataconomy. 2018. Cambridge Analytica Scandal Marks a Shakeup of the Big Data Landscape – Dataconomy. [ONLINE] Available at: http://dataconomy.com/2018/04/cambridge-analytica-scandal-marks-a-shakeup-of-the-big-data-landscape/.
[x] https://www.stableresearch.com.au/news-resources/market-research-industry-needs-to-meet-the-cyber-security-challenge-head-on/,
[xi] Issuu. 2018. Grit report Q3 – Q4 2017 by GreenBook – issuu, p.34 [ONLINE] Available at: https://issuu.com/researchshare/docs/grit_report_q3_-_q4_2017.
[xii] Access Information, [ONLINE] Available at: http://access-information.com/category/business-research/market-trends/
[xiii] IDC: The premier global market intelligence company. 2018. Big Data and Business Analytics Revenues Forecast to Reach $150.8 Billion This Year, Led by Banking and Manufacturing Investments, According to IDC. [ONLINE] Available at: https://www.idc.com/getdoc.jsp?containerId=prUS42371417.
[xiv] Issuu. 2018. Grit report Q3 – Q4 2017 by GreenBook – issuu, p.66 [ONLINE] Available at: https://issuu.com/researchshare/docs/grit_report_q3_-_q4_2017.
[xv] Callegaro, M. (2017). Importance of Surveys in the era of Big Data [PDF] p5. Available at: https://business.unl.edu/academic-programs/programs-and-degrees/sram/documents/Callegaro.
[xvi] Facebook Newsroom. 2018. Using Qualitative Feedback to Show Relevant Stories | Facebook Newsroom. [ONLINE] Available at: https://newsroom.fb.com/news/2016/02/news-feed-fyi-using-qualitative-feedback-to-show-relevant-stories/.
Privacy is Important to us.