6.Marketing Researcher's Panacea or Pandora's Box(1-19)

The Marketing Review 2004, 4, 27-46 28 Paul M. Gurney, Eleanor Chambers, Lynn Grant, Seema Shah and Michael Sullivan advantages and disadvantages of conducting research using the internet as experienced by market researchers. Whilst Ilieva et al (2002) argue the need for a multi-mode approach to research, where the internet is used as a complementary technology, there has been no empirical evidence to substantiate this perspective; the fundamental advantages and disadvantages of conducting online primary research have remained largely theoretical. Based upon an internet survey of UK research professionals, we attempt to substantiate the theoretical arguments presented in the literature by ascertaining the opinions of research professionals, as well as adding to the studies on response times and response rates.This paper begins by examining the existing literature relating to the use of the internet as a means to conduct primary and secondary market research. Proceeding from this review of the literature, we then present our research design, which is followed by the results of our internet survey which was sent out to every ‘European Society for Opinion and Marketing Research’ (ESOMAR) registered UK market research company. This pilot survey represents the first of its kind and attempts to ascertain whether the theoretical opportunities and challenges of the medium are actually perceived by research practitioners. After a discussion of the results, the final section of the paper presents some conclusions based on the findings of the survey. Using The Internet For Market ResearchThe advent of the digital age was heralded by many as presenting humanity with tools that would enable man to ‘analyze more completely and objectively his present problems’ (Bush 1945: 108) and give individuals access to, and command over, the inherited knowledge of the ages. The contemporary incarnation of the internet is still considered by some to represent an ‘endless frontier’ of information (Maloy 1999: vii), presenting marketing researchers with a medium capable of reaching targeted mass audiences at very little cost (Mann and Stewart 2000). The ability to use the internet as a means to ‘systematically design, collect and analyse information’ (Kotler 2000: 103) has been debated by various authors (e.g. Illingworth 2001; Kumar et al. 2002). Many commentators have argued that the internet, as a personalised mass medium (Negroponte 1995), requires unique data collection methods, each with their own methodological challenges (Dillman 2000). The transferability of traditional research methods, with their inherent ontological and epistemological assumptions, to the online setting has also been questioned given the nature of computer-mediated communication (Illingworth 2001). In order to determine the utility of the internet as a research tool it is essential to evaluate its relative advantages over traditional research approaches and examine the inherent difficulties it presents.It is evident that extant literature has adopted a bifurcated perspective of the efficacy of the internet, with some authors arguing that the internet is a revolutionary tool with the potential to redefine the market research industry (e.g. Cornick 1995; Quelch and Klein 1996; Taylor 2000), whilst others haveThe Internet: Marketing Researchers’ Panacea or Pandora’s Box 29 adopted a more critical perspective (Mitra and Cohen 1999; Illingworth 2001; Kumar et al. 2002). Taylor (2000: 53) argues that ‘online research is fundamentally different’, with the internet representing an ‘enhancing, enabling and transforming technology’ (p.57); a perspective shared by Cornick (1995) who states that ‘a new paradigm in the way research is conducted is emerging’. However, some authors have questioned the emergence of this new research paradigm and challenged the perspective that the internet requires unique methodological approaches, arguing that it represents another medium where traditional research techniques are still applied (McDaniel and Gates 2002; Williams et al. 1988). Despite the salience of these issues, the majority of research relating to the effectiveness of the internet has focused on the more quantifiable dimensions of its application as a research tool.A significant proportion of the literature adopts a cost efficiency perspective of the internet, focusing on the relative advantages in terms of increased response rates and reduced response times of e-mail and web-based surveys (e.g. Bachman et al. 1996, 1999; Dommeyer and Moriarty 2000; Schaefer and Dillman 1998; Smith 1997; Weible and Wallace 1998). Whilst the vast majority of research has focused on the issues of cost and time efficiency, there has also been a burgeoning literature on the theoretical utility of the internet as a research tool, most of which delineates between the conceptually distinct areas of primary and secondary research (Costigan 1999).Primary Market Research Using The InternetOne of the fundamental challenges of conducting primary research on the internet has been identified as the limited respondent base of internet users in terms of the number of users and their demographic and psychographic characteristics (Kenway 1996; Turkle 1996). Internet users represent only 9.57% of the world’s population; even in developed Western economies such as the UK, internet users represent only 57.24% of the population (NUA 2003). Whilst there has been a dramatic increase in the uptake of internet technology since Hess (1995: 16) asserted that the internet is largely a ‘playground for the privileged’, Taylor (2000) highlights that the on-line population is still substantially younger and better educated than the public as a whole. This ‘cyberspace divide’, influenced by economics, gender, culture, ethnicity and language (Loader 1998), is seen to result in sample bases which preclude non-internet users and consequently make it difficult to generalise findings (Illingworth 2001; Kumar et al. 2002). However, internet usage in the UK has risen from just 2% in 1997 to 57.24% in 2002 (NUA 2003) and researchers are increasingly using the medium to acquire information on consumers and markets. Whilst the internet still has a somewhat limited sample base for use as a research tool, it does have the ability to transcend geographic distance, thereby lowering the cost of reaching a ‘global’ audience (Kumar et al. 2002).30 Paul M. Gurney, Eleanor Chambers, Lynn Grant, Seema Shah and Michael Sullivan Based on the theoretical advantages of conducting primary research on the internet and the supposed ‘efficiency’ improvements of the medium, Cobanoglu and Cobanoglu (2003: 487) argue that ‘internet-based surveys may soon replace the traditional survey methods of data collection’. The putative superiority of the internet is also implied by Moscardelli (cited by Felson 2001: 43) who argues that due to the limited sample base of the internet, it is important to continue to use offline methods until internet penetration reaches ‘statistically significant levels to warrant sole use of online marketing research collection’. This perspective implies that once internet usage is representative of the population, the technology has inherent advantages over traditional methodologies. In order to examine this assertion, it is necessary to examine the effectiveness of the internet using the different primary research methodologies discussed in the literature.On-Line SurveysOn-line surveys have received considerable attention in the literature, although the majority of studies have concentrated on the notion of ‘efficiency’: deemed to represent the relative cost and time savings of internet research. Internet surveys can be sent to a large number of respondents simultaneously (McDaniel and Gates 1999), representing a significant cost advantage over mail, where each incremental unit incurs an additional cost. It has been estimated that an e-mail survey costs between 5% and 20% of a paper survey and can be administered in a shorter period of time (Sheehan and Hoy 1999; Weible and Wallace 1998). There is also the possibility of personalising surveys through the use of previous database information at a lower cost than with mail surveys (Grossnickle and Raskin 2001; Kumar et al. 2002). The use of text, images, sound, video and animation are also deemed to be beneficial features of internet surveys, with marketing researchers now able to employ a greater variety of survey designs (Fielden 2001; Weible and Wallace 1998).Whilst many ‘efficiency’ studies of internet research have been contradictory and inconclusive (Weible and Wallace 1998), the majority of research supports the assertion that the internet produces decreased response times in comparison with traditional mail surveys, but are inconclusive with regards to whether it increases response rates (e.g. Mehta and Sivada 1995; Schuldt and Totten 1994; Sheehan and Hoy 1999; Sheehan and McMillan 1999; Smith 1997; Weible and Wallace 1998). Sheehan (2001) identifies that response rates to e-mail surveys have been dropping since 1986 and argues that there are many factors influencing the level of response. Primary factors are identified as being the salience of the topic with the sample base, survey length and the number of follow up contacts and the year of the survey; issues frequently overlooked in many studies. The increase in ‘spam’ has also been found to have a negative effect on survey responses (Sheehan and Hoy 1999); the increasing numbers of unsolicited e-mails are deemed a nuisance and intrusive, with many potential respondents ignoring invitations to participate (NUA 2003).The Internet: Marketing Researchers’ Panacea or Pandora’s Box 31 It has been determined that respondents are more likely to participate in surveys through the use of incentives (Cobanoglu and Cobanoglu 2003), which are increasingly being utilised by market research companies in an attempt to increase response rates. However, Ilieva et al (2002) suggest that this may have negative effects as respondents might be tempted to distort data; providing hurried and inaccurate information may potentially result in a ‘carnival of misuse and abuse’ (Tortorello 2002: 20).This is supportive of suggestions from Nancarrow et al (2001) who propose that improved accessibility of survey research to new sectors such as University students, public sector departments, small and medium size businesses, may have a potentially damaging effect on the research industry. Although internet based surveying raises new methodological challenges, Dillman (2000: 400) argues that ‘no other method of collecting survey data offers so much potential for so little cost’.Online Focus GroupsWhilst traditional focus groups are often deemed to be expensive and time consuming, Deal and Hodson (1997) identify a number of potential advantages of conducting online focus groups. They suggest that it encourages ‘hesitant’ participants to speak more freely, whilst moderating the hegemony of dominant respondents and reducing the problem of participants all speaking at once. However, Curasi (2001) suggests that online group dynamics occur very differently in a computer-mediated environment. The absence of an interviewer results in an environment devoid of physical and social stimuli (Illingworth 2001), increasing the difficulty of building a trusting and intimate relationship with respondents which inevitably limits the role of the researcher as the ‘major research instrument’ (Burgess 1984: 218). The difficulty of controlling and directing online focus groups can be seen to limit the information elicited from the interview, with respondents often giving only short and superficial answers (Scholl et al. 1999).Ethnographic StudiesSome authors have gone as far as arguing that the internet may also be used to conduct ethnographic studies or ‘market-oriented netnography’ (Kozinets 2002: 63), where market researchers are able to obtain in-depth and culturally attuned consumer information (Catterall and Maclaran 2002). Adopting the perspective that online consumer groups are essentially discursive spaces characterised by ‘shared, mutable communicative protocols that facilitate intersubjective understanding’ (Worthington 1996: 67), it could be argued that the internet is a valuable research tool enabling marketers to access these transitory and polyphonic domains. Although the internet is increasingly being utilised as a means to enable audio and video communication, the majority of discursive interactions are textual, with researchers able to observe, record and analyse consumer dialogues. The plethora of observable online cultures allows researchers to study specific market segments, observing sub-cultural narratives and facilitating the32 Paul M. Gurney, Eleanor Chambers, Lynn Grant, Seema Shah and Michael Sullivan identification of emerging niche markets (Catterall and Maclaran 2002). Practices such as ‘lurking’ (non-participant observation) provide the possibility for market researchers to immerse themselves into a community whilst remaining anonymous. Catterall and Maclaran (2002) argue that such techniques enable the researcher to learn the rules of the community by observing the ‘web of personal relationships’ (Rheingold 1993: 5), thus enabling the collation of richer and more insightful data.Whilst the concept of ‘netnography’ has been proffered as a viable research technique (e.g. Catterall and Maclaran 2002; Kozinets 2002), a more critical perspective of online ethnography would suggest that the ability to examine consumer’s ‘imbricated layers of cultural meaning’ (Thompson and Troester 2002: 550) is highly limited. Given Geertz’s (1973) assertion that one of the qualities of ethnography is the ability to convey a sense of ‘being there’, the physical dislocation of the researcher from the respondents limits the extent to which a ‘deep’ and ‘contextual’ insight (Dyer and Wilkins 1991: 613) can be achieved. As a result of not ‘being there’, the written word becomes the primary source of data for the ethnographer; without actively observing consumers’ behaviour and their emotional responses, the notion of ethnographers being able to ‘infiltrate the expressional universe of the other’ (Clifford 1983: 100) is somewhat problematic.The transient nature of virtual communities and the inability to determine identities are also deemed to be pertinent issues. Considering the postmodern perspective of individualism and social fragmentation (e.g. Cova 1996; Firat and Schultz II 1997), the internet can been seen as a technology that facilitates multiple (re)constructions of the self. Whilst remaining anonymous, individuals are able to author a multiplicity of identity narratives, undermining the notion of researching a ‘real and unitary self’ (Turkle 1996: 267). It is possible for members of virtual communities to have multiple and coexistent identities that can be created and discarded as they choose, creating the possibility of unknowingly researching multiple narratives of the same individual (Catterall and Maclaran 2002).Secondary Research Using The InternetIt has been argued by Kumar et al. (2002) that whilst conducting primary research presents many methodological challenges, the forte of the internet is probably its advantages in researching secondary information. Secondary research on the internet is often cited as a primary advantage of the technology, principally because of its convenience and low cost, the speed at which data can be collected, the vast amount of information available and the availability of online information that is often more current than data published in hard copy (Burns and Bush 2000; McDaniel and Gates 2002). The internet is also particularly suited to the acquisition of secondary data since many of the traditional sources of information have migrated online (Chaffey et al. 2003); these include internal company data, online market information and services such as Mintel and Keynote, trade directories,The Internet: Marketing Researchers’ Panacea or Pandora’s Box 33 government statistics, and trade journals (Bickerton et al. 2000).The internet can also be useful as a source of information on competitors, demographic trends, socio-cultural data and information on legal or political environments. It has been argued that prior to the advent of the internet this information was either completely unavailable or very difficult to obtain. Specifically with regard to competitor information, companies now have instant access to competitors’ organisational objectives, product ranges, product initiatives, pricing and promotional activities (Coupey 2001).The volume of information available on the internet and the low cost of accessing information may preclude the need to conduct primary research, as Castleberry (2001: 195) questions ‘why create new knowledge using primary data collection if the knowledge already exists and can be found using secondary sources?’. Whilst the internet may represent a ‘treasure trove of potential research insight’ (Coupey 2001: 201), it has been argued that there is a greater need to critically evaluate internet sources than with more established off-line resources (Coombes 2001). Largely due to issues of authorship and authenticity (Mitra and Cohen 1999), transparency and obsolescence of data, accuracy and plausibility are particularly pertinent issues in internet research (McDaniel and Gates 2002). Given the vast array of information available on the internet, acquiring meaningful data and valuable marketing information can be problematic. It has been argued that the multiplicity of sources can result in researchers experiencing ‘information overload’ (Ó’Dochartaigh 2002: 7), sieving through the ‘shifting chaos’ of the internet (p.114).Research DesignIn order to ascertain the perceived utility of the internet as a research tool, as well as examining online survey response rates and times, it was deemed appropriate to use the internet as a means to acquire this information. The market research industry was utilised as a sample base given its role in providing clients with detailed marketing information combined with the experience of research professionals in using multiple methodologies to acquire information. Whilst the sample is not necessarily typical of marketing research, which often focuses on consumer groups, it does serve to provide information on response rates that can be compared with previous studies. Preliminary secondary research was conducted in an attempt to determine the current level of usage of the internet by market researchers. The ESOMAR website () was used to determine the proportion of marketing research agencies currently using the internet for primary research. A web-based survey was designed to examine the types of research conducted on the internet, as well as determining the perceived advantages and disadvantages of the internet compared with traditional methodologies. The survey was split into two sections: the use of the internet as a primary and secondary research tool and a section designed to ascertain the attitudes of market researchers to the internet. Questions in the34 Paul M. Gurney, Eleanor Chambers, Lynn Grant, Seema Shah and Michael Sullivanfirst section were based on the theoretical arguments advanced in theliterature and examined the utilisation of the internet for primary andsecondary research. Respondents were asked to identify the types of primaryand secondary research conducted as well as their perceived advantagesand disadvantages of the medium. In this section respondents werepresented with a list of possible responses as well as an option to provide‘other’ responses and further comments.The second section of the survey provided a list of ten statementspertaining to the efficacy of the internet as a research tool. A seven pointLikert scale was employed to allow respondents to indicate their perceptionsof the internet on many substantive issues relating to the utility of themedium. An e-mail with a link to the survey was sent to all 146 UK marketresearch agencies registered with ESOMAR inviting them to participate in thesurvey. Responses were automatically recorded in a log file which generateda database of the responses to the survey, stored the time of the surveycompletion as well as IP addresses in order to ensure that identical multiplesubmissions were identifiable.ResultsTable 1. Response Rates and Response TimesDelivery Statistics Response by DaysSent out 146Day One 41Undelivered 4DayTwo14Rejected as Spam 1Day Three 4Total 141DayFour0 Responses 59Total 59 Response Rate 41.80%Response TimesNumber of Respondents Total Responsetime (minutes)Average ResponseTime (minutes)Average Responsetime (hours)Day One 41 1558 38 0Hours38MinutesDay Two 14 15652 1118 18Hours38MinutesDay Three 4 10780 2695 44 Hours 55MinutesTotal 59 27990 474.4 7 Hours 54MinutesOf the 146 e-mails sent to UK based market research companies registered with ESOMAR, 4 were returned as undelivered and 1 was rejected as ‘possible spam’ representing a delivery of 141 e-mails. 59 responses wereThe Internet: Marketing Researchers’ Panacea or Pandora’s Box 35 received, representing a response rate of 41.8%, comparatively high considering the disparate and contradictory findings of research relating to response rates of e-mail surveys and web-based surveys (Sheehan 2001; Smith 1997; Weible and Wallace 1998). This finding supports Sheehan’s (2001) assertion that the salience of the subject is one of the primary factors influencing response rates given the relevance of the topic to the survey sample. The response rate may also be relatively high due to the fact that this research focuses on market research companies as opposed to consumers. One respondent argued that the internet represents a powerful ‘high penetration tool in business research’, offering ‘statistically significant results’ that are ‘less likely to be achieved in consumer research’.Contrary to many findings regarding response times to internet research, the average response time of each respondent was 7 hours and 54 minutes, with 65% of responses received within 3 hours; substantially less than the 7.4 days cited by Weible and Wallace (1998). This is likely to be due to the pervasive expansion of the internet and the increased usage of e-mails since many of the earlier studies (e.g. Bachman et al. 1996; Kiesler and Sproull 1986; Schaefer and Dillman 1998; Weible and Wallace 1998); corporate LAN’s and permanent internet connections are ubiquitous in many corporate environments, with more employees able to access the internet, facilitating prompt responses to e-mails.Uses and Perceptions of the Internet as a Primary Research Tool (Appendix I)Of the 59 respondents, 76.3% stated that they used the internet for primary research which is comparable to the figure of 69.9% of all UK based market research companies using the internet for primary research (ESOMAR 2003). The vast majority of market research companies conducting primary research over the internet used web-based surveys (72.9%) compared with 50.9% who used e-mail surveys. This may be due to the increased ability to utilise ‘multi-media opportunities’ (respondent, multinational market research company), visual stimuli and the creation of databases, supporting the arguments of Weible and Wallace (1998). Although Sheehan’s (2001) results indicate that responses to e-mail surveys have dropped significantly since the first study in 1986, it appears that e-mail surveys are still considered to be a viable means of acquiring data given that they remain widely utilised by market researchers.Despite the increasing attention given to the value of phenomenological approaches to internet research such as on-line participant observation (Kendall 1999), discourse analysis (Jones 1999) and ethnographic studies of virtual communities (Catterall and Maclaren 2002; Kozinets 2002), relatively few of the market research companies used on-line focus groups (13.6%) and only a small minority actively researched virtual communities (3.4%). This would appear to support the perspective that the internet is less suited to interpretive research; the ability to ‘penetrate another form of life’ (Geertz 1988: 5) and achieve an ‘intuitive and empathetic understanding of the other’36 Paul M. Gurney, Eleanor Chambers, Lynn Grant, Seema Shah and Michael Sullivan (Humphreys et al. 2003: 10) appears to be extremely limited. This perspective is further supported by the 39.0% of respondents who stated that the difficulty of obtaining detailed qualitative data is a primary disadvantage of internet research.Cost efficiency was identified as the primary advantage of internet market research with 71.2% of responding UK market research companies supporting the findings and arguments presented in the literature (Kumar et al. 2002; Mann and Stewart 2000; Sheehan and Hoy 1999; Weible and Wallace 1998). The unobtrusive nature of internet surveys was also deemed to be an advantage of the internet as respondents can participate at their own convenience (Kumar et al. 2002), with 54.2% stating this factor to be an advantage of internet research. The global reach of the internet was also deemed to be advantageous with 49.2% of respondents stating that this was a primary benefit of the internet as a research tool. An additional advantage was the ability to conduct research at great speed with the turnaround time for results being much quicker than using other techniques.Of the 59 respondents, 72.9% stated that the limited sampling base of the internet, in terms of user demographics and psychographics was a primary disadvantage of the medium. This data supports the perspective that the internet has acute methodological challenges in terms of its demographic and psychographic composition, resulting from the existing social divisions of those with, and those without, access (Turkle 1996; Kenway 1996). The ability to initially engage with respondents was also identified as a disadvantage of the medium with 30.5% stating that building the respondent base was a significant challenge, although as one respondent identified, this challenge is not unique to the internet. Also identified as a methodological issue was the ability of market researchers to determine the identity of the respondent; as Porter (1997: xi) argues, internet users have ‘no bodies, no faces, no histories beyond that which they wish to reveal’. Coomber (1997) also argues that e-mails may be forwarded to other individuals outside the sampling base which raises issues of ‘sample integrity’ (Managing Director, international consumer research company).Uses and Perceptions of the Internet as a Secondary Research Tool (Appendix II)The assertion of Kumar et al.(2002), that the internet’s forte is its ability to facilitate the acquisition of secondary data, is supported by the fact that 89.83% of respondents use the internet as a means to conduct secondary research. Whilst compiling information on products was found to be the greatest use of the internet (83.2%), respondents had a high propensity to use the internet for a number of functions associated with secondary research. The primary advantages of conducting secondary research on the internet were identified as the speed at which information can be acquired (78.0%), the vast array of information available (76.26%) and the ability to search through this information (71.2%). The low cost of information acquisition was also highlighted as a significant advantage with 52.5% of。