It could be said that respondents are the capital of the market research industry. Developing a relationship that supports and increases the value of that respondent both to the industry and the client should be an ongoing priority for market research professionals. As we continue to integrate technology and online research methods as standard practice it is important to shape that development in a knowledgeable way. While technology is an enabler it also in some ways diminishes control over the respondent relationship in terms of external environmental factors and engagement levels.
During December 2013 Stable Research conducted a survey that explored the opinions, likes and dislikes of our panel members. Our findings closely correlate with lifestyle preferences that are reflected through findings that show research timing is more likely to encourage engagement over incentives. The respondent survey findings also highlight that participants enjoy focus or discussion group research over other methods and that the research topic also substantially influences the decision to participate. In contrast, respondents do not enjoy completing pre-qualifying questionnaires and never being contacted, completing surveys and failing at the end as well as irrelevant or repeated questions. As we hurtle headlong into the development of research methods that embrace technology we must also learn from the end user experience to ensure that we harness innovation in new and exciting ways.
PARTICIPANTS, PREFERENCES AND PERFORMANCE
Market research firms continue to place enormous effort into developing new online products and services, and rightly so. But, is enough thought going into the opinions of respondents when it comes to their engagement and participation levels? If the industry is to continue to provide quality products and services then surely that development is inextricably linked to finding out not only how respondents engage, but their likes and dislikes associated with various survey methods.
In commenting on their study into online survey respondent engagement, Downes-Le Guin, Baker, Mechling and Ruylea (2012) noted that ‘the keys to greater survey engagement lie not in graphical enhancements or greater interactivity in presentation of survey questions, but rather in dealing more effectively with the fundamental components of respondent burden that survey methodologists have long recognised: survey length, topic salience, cognitive burden (i.e. poorly written or hard to answer questions) and frequency of survey requests’ (p. 18 ). At present there is a concerted focus on the development of online research methods and tools, but what about other research methods? Do other research methods provide greater levels of satisfaction and in turn engagement and could some elements of popular research methods be more effectively integrated into online market research developments?
OUR APPROACH
While there have been numerous studies over the years that have focused on various aspects and components of the market research industry, in November 2013 Stable Research approached the Research News Advisory Group with the idea of conducting research that focused on our panel and their experience and preferences in dealing and engaging in the market research process. Stable Research has a panel of 100,000 members matching with Australian Bureau of Statistics (ABS) Census 2011 distribution of gender and age (ABS, 2012, 3235.0. Between 8 November and 14 November 2013 we conducted an online survey of 10,000 panel members and did not apply any quotas because we wanted a true reflection of our panel. Survey respondents were approached via an email asking them ‘Help shape the future of Research!’ Incentives to participate were offered.
Stable Research received 940 responses with a large majority responding via a computer, rather than a tablet or mobile phone. In 2012-2013 ABS statistics highlighted that men and women accessing the internet were almost even at 84 percent and 83 percent respectively.’ (ABS, 2013, 8146.0). In this instance more women than men responded to the survey at 62 percent and the profile was slightly older than the general ABS Australian population average of approximately 37 years (ABS, 2013, 8146.0). Respondents were more likely to be married with children at home than a population survey would achieve and there was a slight skew to Sydney. Respondents replied from metropolitan and regional areas with a greater number of respondents replying from regional Queensland rather than Brisbane.
QUESTIONS ASKED
While our respondent survey was only online, it also sought opinions about the type of research that people had participated in, their ongoing commitment levels, changes in participation rates and their levels of respondent enjoyment for various research methods. Questions were asked to ascertain why respondents participate in some research and not others; survey preference ranking and enjoyment level ranking. The tools that respondents use when participating in surveys and the influence of those tools on engagement levels were also explored.
Burden levels were ascertained through questions associated with potentially the biggest frustrations when answering surveys, opinions about pre-tasks or homework and incentives or payments related to survey type. Timings and invitations to participate were also explored. Survey topic preference as well as gender, age bracket, household structure and place of residence rounded out the survey questions.
The respondent survey findings were then provided to McNair Ingenuity Research so they could independently analyse the results and report back to us identifying key trends.
PREVIOUS RESEARCH PARTICIPATION
A large percentage of respondents had participated in an online survey before at 84 percent, at the same time 66 percent had participated in focus or discussion groups and online community research participation was at 17 percent.
Online survey participation rates were similar across all age groups. In the 51 to 60 year age group 72 percent indicated they had participated in focus or discussion groups in comparison to 45 percent of 18 to 25 year olds. The survey revealed that 28 percent of people in metro areas had completed individual interviews in comparison to 17 percent of respondents in regional areas. Male and female participation rates for online surveys were somewhat equal. There were a relatively low percentage of respondents who had participated in online communities before when compared to other research methods but this was probably influenced by the fact that online communities are a reasonably new method of research.
On average respondents receive 1.6 invitations per day with 18 to 25 year olds receiving less than one invitation per day. Respondents also indicated that on average they replied to 75 percent of the invitations that they received.
WHAT TYPE OF RESEARCH DO RESPONDENTS LIKE TO PARTICIPATE IN?
It is no secret that people are more likely to engage with something if they enjoy it. From a market research perspective enjoyment can be influenced by a number of variables. Enjoyment can stem from the type of research, research topics and incentives, down to the type of refreshments that are provided and parking availability at the market research venue.
The respondent survey highlighted that focus or discussion groups, online communities, and online surveys as well as in depths resulted in the highest levels of enjoyment. Enjoyment levels for discussion groups were relatively equal for male and female respondents with 41 to 50 year olds as well as single people with children at home expressing the highest level of enjoyment. Focus groups in market research are nothing new. They have played an important role in both marketing and political decision making for many years and respondents can be empowered by their involvement in the decision making process. Gibbs notes in her paper about focus groups that ‘the opportunity to be involved in decision making processes, to be valued as experts, and to be given the chance to work collaboratively with researchers can be empowering for many participants.’ (Gibbs, 1997).
Levels of enjoyment for online surveys were highest for participants in the 26 to 30 age bracket, yet 77 percent of the 61+ age group also indicated that they enjoy participating in online surveys. However, the 61+ age group was from a smaller base, and biased by the fact that they are already part of on an online panel. Similarly, enjoyment levels for participation in online communities were highest in the 41 to 50 and the 51 to 60 and brackets. It is interesting to note the relatively high enjoyment levels of older Australians who had participated in online market research survey. Recent Australian Bureau of Statistics research indicates that persons in the 15 to 17 age group had the highest proportion of internet users at 97 percent in comparison with older persons aged 65 years who had the lowest proportion of internet users at 46 percent (ABS, 2013, 8146.0). This would suggest that those engaged in online research are comfortable using a computer and navigating the online research environment regardless of age. Levels for 61+ and online engagement do drop away when investigating engagement via a mobile phone or tablet.
Phone polls, telephone research and surveys in shopping centres all elicited the lowest levels of enjoyment for research participants. Phone polls provided the lowest level of enjoyment for younger people with only 33 percent of 18 to 25 year olds indicating that they enjoy telephone research in comparison to 51 percent of 61+ year olds.
While 28 percent indicated that they had previously participated in surveys in shopping centres/train stations or other public locations, 27 percent of males and 21 percent of females indicated that they do not enjoy participating in those types of surveys in comparison to 2 percent of males and 1 percent of females that do not enjoy focus or discussion groups. The highest level of enjoyment in shopping centre/train station or other public location research was from the 61+ age group.
TIMING VERSUS INCENTIVES
One of the recommendations by Cobanoglu and Cobanoglu (2003) in their study into the effect of incentives in web surveys supported the use of incentives when conducting online surveys to achieve higher response rates. The use of incentives and incentive schemes for market research participation are commonplace. Incentives can include monetary sums, ‘points’ earned to cash or products. Focus groups and interviews provide an average incentive payment of eighty dollars for two hours. Discussions about sensitive health topics or large financial investments also tend to provide higher incentives levels. But, do respondents feel that the level of incentives provided are adequate for the time and effort that they put into answering the various types of surveys? To ascertain their opinion the respondent survey explored participant opinions about the level of incentives provided for the various forms of research.
While 35 percent of respondents felt that incentives for online survey participation were adequate, 38 percent felt that incentives were too low. A significant number of younger people felt that incentives were too low when compared to older age groups. The respondent survey found that 27 percent of 18 to 25 year olds believed that incentives for online research were good. In comparison 12 percent of 61+ year olds believed they were good. While older age groups said they enjoyed online surveys their answers about incentives may be influenced by the amount of time it took them to navigate the online survey when compared to younger age groups. In contrast to online surveys, the vast majority of respondents felt that incentives for discussion groups were good or adequate. Incentives for participation in online communities, blogs and forums also elicited a response where 22 percent felt incentives were adequate but 19 percent believed they were too low. In terms of participation rates, 49% of respondents indicated that they had not participated in this type of research. The relatively infancy of online community research is highlighted in the Research Industry Council of Australia’s report Australian Markets and Social Research Industry Summary 2012 that notes in 2012 only 2 percent of the total research turnover of Australian market research companies was achieved through online (client owned) community panels. (Research Industry Council of Australia, 2013)
Incentives are relevant to survey participation yet it was timing that trumped incentives when it came to the decision whether to participate in some research and not others. The respondent survey showed 76 percent considered that lack of time was the reason that they participated in some research and not others compared to 52 percent who engaged because of incentives and 42 percent for the survey topic.
Recent findings of the OECD Better Life Initiative research highlight that while Australia ranks at the top in civic engagement and above average for environmental quality, health status, housing, personal security, jobs and earnings, education and skills, subjective well-being, social connections and income and wealth, it ranks in the bottom 20 percent in terms of work-life balance (OECD, 2014). The survey respondents could be reflective of broader OECD findings that highlight poor work-life balance results for Australians and therefore a lack of disposable time.
If lack of time is significant then answers related to the time that respondents would be more likely to answer a survey become more relevant. The respondent survey showed that the most popular time for survey participation was weekday evenings at 61 percent, followed by Saturday day time at 54 percent and Sunday daytime at 52 percent.
WHAT DO RESPONDENTS LIKE ABOUT ENGAGING IN MARKET RESEARCH?
As a result of their study focusing on leverage salience theory, Groves, Presser and Dipko (2004) noted that persons cooperated at higher rates to surveys on topics of likely interest to them. Participant interest in a topic was highlighted as the most important factor in relation to enjoyment levels in group discussions.
More male than female respondents indicated that giving their opinion was the first preference when asked about the reason why they attended market research. Providing an opinion proved popular regardless of the survey method. It was most popular during in-depths and street/central location surveys at 37 percent, followed by telephone and phone polls at 36 percent, online survey and groups at 35 percent and online community research at 33 percent. Incentives ranked highest for 18 to 25 year olds and married/de-facto couples with no children. Enjoyment related to finding out about new products and services also rated well with this group. It appears that people do not really attend research to meet other people as enjoying the market research process and meeting other people achieved the lowest ranking responses.
The Nielsen Top Media Advertisers Australia for 2013 report noted that ‘the top 10 advertiser groups in Australia spent an estimated $1.2 billion in 2013. The categories which were most representative in this elite list of top 10 advertiser groups were a combination of retail, motor vehicle, government, FMCG and communication categories’ (Nielsen, 2014).
If these categories are viewed in terms of research topics that people enjoy most we can see that food, health and other grocery items top the list. Demographic variables across those findings highlight that more women than men prefer surveys about food and other grocery items. Older Australians 61+ enjoy surveys about food, health and government or social issues. As the age of respondent groups increased so did the percentage of those who enjoyed surveys about government or social issues. Enjoyment levels for surveys about alcohol and non-alcoholic beverage were quite similar except for the 18-25 age groups who enjoyed surveys about alcohol more than other age groups. Surveys about financial, technical and automotive had the lowest enjoyment levels, especially for female respondents. When answering market research surveys 29 percent of respondents said that they had enjoyed completing surveys about food online and by telephone, followed by online community and groups and street/central location at 28 percent. The percentage of respondents who enjoyed surveys about food across all survey methods was more than double that of health and other grocery items. This could support an argument that respondents engage with surveys that are of interest to them, regardless of survey method.
WHAT FRUSTRATES RESPONDENTS WHEN PARTICIPATING IN MARKET RESEARCH?
A lot of effort goes into ensuring that survey design not only engages participants but also promotes quality. Issues associated with poorly written questions, survey length and technological skills are all relevant when considering engagement with the end users. When asked to name their top three frustrations when answering surveys 19 percent of respondents said that completing prequalifying questionnaires and never being contacted was their biggest concern. This was followed by completing surveys and failing at the end and irrelevant/repeated questions at 17 percent. Survey length was the biggest frustration for 16 percent of respondents followed by confusing questions such as double negatives at 12 percent, asking questions that you have already given to a panel at 11 percent and poor questionnaire design at 9 percent.
It probably does not come as a huge surprise that survey length was particularly problematic for the millennial 18 to 25 years age group and single people with children at home. A larger percentage of age groups 41 and above expressed frustration at completing prequalifying questionnaires and never being contacted again, this was also particularly relevant for the 61+ age group.
It is interesting to note that the percentage of respondents who expressed frustration at completing pre-qualifying questionnaires and never again being contacted was fairly equal across online surveys, groups, telephone, street/central locations and phone polls. When asked about their top three frustrations, online communities, phone polls and telephone research came out on top.
The percentage of people that highlighted frustration with poor questionnaire design was less than half when compared to completing pre-qualifying questionnaires and never being contact across all types of research. That finding could highlight the fact that the concept of poor survey design is more relevant to the development of industry standards and best practice rather than respondent satisfaction and engagement.
THOUGHTS ABOUT PRE-TASKS OR HOMEWORK
Pre-tasks or homework often form part of the research process to assist with research engagement and also to help facilitate discussion that delves deeper into respondent insight. Respondents may be requested to complete a task prior to their participation in a group discussion or an interview. Respondents also often receive additional incentives for completing pre-tasks or homework depending on the length and complexity of the task.
When responding to questions about pre-tasks or homework 60 percent of respondents felt that it helped with discussion during research while 15 percent indicated that they felt it was a challenge to complete. The 51-60 year olds felt that pre-tasks and homework helped with discussion during research at 67 percent whereas 54 percent of 18-25 year olds felt that this was the case. While 19 percent of 18-25 year olds could not see any relevance in pre-tasks or homework only 4 percent of 26 to 30 year olds believed this was the case.
HOW RESPONDENTS HAVE ENGAGED IN THE PAST
A total of 46 percent of respondents have participated in research utilising mobile phones with 60 percent of 18-25 year olds indicating they have done so in comparison to 28 percent of 61+ year olds. In relation to household structure, 58 percent of married/de-facto couples with no children indicated that they had participated in a survey using a mobile phone. Smart phone ownership will no doubt influence these statistics going forward. The Australian Communications and Media Authority have reported that 11.19 million people owned a smart phone at May 2013, up 29 percent from May 2012 (ACMA, 2014).
Tablet devices were used by 34 percent of 31 to 40 year old respondents when participating in surveys. While app usage for survey response was under 17 percent for most groups, it is significant to note that 24 percent of 18-25 year olds indicated that they had used an app to participate in a survey.
In relation to tools used when participating in research, 34 percent of respondents indicated that they had not used a mobile phone, tablet, videos, apps or blogs when participating in a survey with 46 percent in the 61+ age group and 45 percent in the 51-60 year old age group indicating they had not used any of the tools.
WILLINGNESS FOR ONGOING MARKET RESEARCH ENGAGEMENT
A large majority of respondents indicated that they would continue to participate in most forms of research. While participants in the respondent survey could choose multiple options a willingness to participate in online surveys proved most popular at 96 percent across all age groups, household structures, location and gender. This finding could indicate that online surveys meet the ability to satisfy time requirements of participants. Continuing or starting to participate in focus or discussion groups was also significant, especially for the 41 to 50 year age group.
A willingness to participate in surveys in shopping centres/train stations or other public locations elicited a positive response from less than 50 percent of overall respondents. Willingness to participate in online communities was also less than online surveys. Pitta and Fowler (2005) have noted in relation to online communities that lack of trust was the one key obstacle affecting responses and information sharing. In contrast, the use of pseudonyms and the public sharing in the discussion tends to build trust over time. There may also be a factor of the unknown in relation to future engagement with online communities.
WHERE TO FROM HERE?
The respondent survey provides valuable insight that can in turn guide future developments that enhance response and engagement levels. Regardless of technological development, respondents still find face to face engagement through focus groups to be one of the most enjoyable forms of research.
At this stage a willingness to start or continue to participate in online surveys far outweighs the percentage of respondents who want to engage through online communities. Yet, it is online communities that could best emulate the two way communication experience of a face to face environment. This lack of willingness to engage may be associated with privacy concerns or a lack of understanding about the online community research process.
While the research shows that a large percentage of respondents continue to engage via personal computer, the proliferation of smartphones and tablet devices will ultimately influence engagement levels using those devices across all demographics. Older respondent groups show a willingness and enjoyment with online participation and it will be relevant to encourage this group to engage as smartphone and tablet device proliferation increases. The development of effective research methods that utilise mobile devices may also go some way to meeting the needs of time poor respondents. The future development of the use of mobile devices must also take into consideration a best practice approach to engagement levels related to where and when the respondent is completing the survey as well as outside influences and distractions.
The trend for online engagement also needs to be reviewed in terms of incentives provided, as the respondent survey indicates that two in five respondents believe incentives for online surveys are too low. This may have something to do with the perceived value associated with online engagement over face to face research methods.
The survey findings also showed that while over half of the survey respondents indicated that their research participation had increased or stayed the same, nearly 41 percent indicated that their participation rates over the past ten years had declined, particularly for groups. Time pressures and also the industry’s tendency to develop and market online research methods may see group survey participation decline even further over the coming years.
If engagement is linked to enjoyment, as our research indicates, then the development of new and innovative research methods must draw on elements of in-depths and group research that promote a sense of participation, personality and community. At the present time online communities are the obvious vehicle through which to explore a more effective approach.
REFERENCES
Australian Bureau of Statistics, (2012). Population by age and sex, regions of Australia, 2011. 3235.0
Australian Bureau of Statistics, (2013). Household use of information technology, Australia, 2012-13. 8146.0
Australian Communications and Media Authority, (2014). Australia’s mobile digital economy – ACMA confirms usage, choice, mobility and intensity on the rise. Retrieved on May 2014 from http://www.acma.gov.au/theACMA/Library/Corporate-library/Corporatepublications/australia-mobile-digital-economy.
Cobanoglu, C., & Cobanoglu, N. (2003). The effect of incentives in web surveys: application and ethical considerations. International Journal of Market Research, 45 (4), 475-485
Downes-Le Guin, T., Baker, R., Mechling, J.,& Ruylea, E. (2012). Myths and realities of respondent engagement in online surveys. International Journal of Market Research, 54 (5), 1-21
Gibbs, A. (1997). Focus groups. Social Research Update, University of Surrey. Retrieved on May 2014 from http://sru.soc.surrey.ac.uk/SRU19.html
Groves, R. M., Presser, S., & Dipko, S. (2004). The Role of topic interest in survey participation Decisions. Public Opinion Quarterly, 68 (1) Retrieved on May 2014 from http://poq.oxfordjournals.org/content/68/1/2. full#sec-12
Pitta, D. A., Fowler, D., (2005). Online consumer communities and their value to new product developers. Journal of Product & Brand Management, 14 (5). Retrieved on May 2014 from http://home.ubalt.edu/ntsbpitt/Onlinepercent20Consumerpercent20Communities.pdf
Research Industry Council of Australia (RICA), (2013). Australian Market and Social Research Industry Summary 2012, July 2013, p10
The Nielsen Company, (2014). The Top Media Advertisers January to December 2013. Retrieved on May 2014 from http://www.shopnielsen.com/downloads/nielsen_top_ad_report_march_2014_sampler.pdf
The Organisation for Economic Co-operation and Development, (2014). OECD Better Life Initiative Retrieved on May 2014 from http://www.oecd.org/general/Better-life-index-2014-country-reports.pdf
Click here to download the full article as a pdf
Privacy is Important to us.