Online surveys ‘more hype than action’

With public disaffection for telephone interruptions at meal time and solid online penetration across the country, surveying via the Internet seems to be a natural. But despite predictions that it would revolutionize the industry, it hasn't exactly taken Canada's research community by storm.

With public disaffection for telephone interruptions at meal time and solid online penetration across the country, surveying via the Internet seems to be a natural. But despite predictions that it would revolutionize the industry, it hasn’t exactly taken Canada’s research community by storm.

The national average for 2000 is that only 3% of all market research is conducted online, reports Norman Mould of Vancouver, president of CAMRO (The Canadian Association of Marketing Research Organizations). The largest chunk of research companies’ work is still devoted to telephone surveys, at 59%. Though 2001 figures have yet to be tabulated, he says online research is likely to increase only marginally.

The big hurdle with the Internet seems to be questions about just how representative of the total population it is. In other words, can business really rely on data sourced from online surveys?

Sixty per cent of Canada’s population is online – higher than in the U.S. – but some researchers are still uncomfortable leaving out that other 40%. There’s also the demographics issue. As Bob Collins, director of customer insights for the Toronto-based Liquor Control Board of Ontario puts it: ‘A hotmail address could be anyone from a kid to someone from Timbuktu.’

The low demand for Internet research is no surprise to Mike Gadd, president of Gadd International Research, a mid-sized research company in Toronto that mainly works with Fortune 500 companies.

‘We would love to do more Internet surveys,’ says Gadd, but he finds his clients are still ‘too skeptical’ of computer-assisted research. There are doubts about the Internet’s representation and therefore accuracy, but he also explains it as ‘Canadian reticence.’

However, Steve Levy, president of Ipsos-NPD in Toronto, likens it to any new technology: Internet research may be in its infancy, but it still has a bright future. Among the first companies to dive into Internet surveys, Ipsos-NPD currently does 17% of its surveys online.

To assuage their own doubts, over the last few years some market research suppliers have made ‘fairly major investments into Internet research so that they can control the nature of the sample and know who their respondents are,’ says Mould.

One of a handful of companies investing in developing Internet methodologies is Ipsos-NPD, and Levy explains the four main approaches to conducting online surveys.

The first, and least-favoured, is e-mail surveying via purchased database lists. ‘In most cases, you don’t know where the list comes from or how clean it is,’ says Levy. There can also be software compatibility problems when using e-mails enhanced with more elaborate formatting.

The next three methods are all Web-based, ‘where most good surveys are executed,’ says Levy. The first – site-based studies – features pop-up banners at appropriate Web sites that invite surfers to take part in a survey. If the answer is yes, the respondent clicks on a link to the survey site.

The second Web-based approach is called a subscriber or customer study. For example, a bank which has online customers provides a list of those customers to an Internet researcher in order to survey customer satisfaction levels with their online banking system.

The last, and most favoured approach for many applications, is panels. However, only a limited number of companies use this method because it is the most expensive. A more elaborate infrastructure is required, including servers, software engines and an expert labour pool to carefully recruit, select and manage panels.

Once a panel is in place, however, it can be used for a variety of studies. Furthermore, Levy notes, because respondents have already expressed interest in being part of a panel, the response rate is very high – 50% to 60% in many cases – whereas other research rates are stuck in the single digits.

Ipsos-NPD has the largest statistically representative online panel in Canada, with 55,000 members. As long as panels ‘look like Canada,’ as Levy puts it, clients can rely on the survey results.

‘Our feeling is that the way to do it [Internet research] is to build a panel,’ agrees Peter Greensmith, executive VP at Toronto-based Market Facts of Canada, one of the main Internet suppliers. ‘It works the same as a mail panel,’ which is a tried-and-true research method, he says.

Some companies are trying to overcome representation fears by conducting ‘parallel testing’ to measure the Internet’s effectiveness. Getting its feet wet in this area, Toronto’s Thompson Lightstone & Company completed a small pilot study last fall – a simultaneous Omnitel online and phone survey. Appropriately, it investigated attitudes and beliefs about the Internet. (Omnitel is TLC’s brand name for an omnibus survey including questions from several different companies, thereby spreading the costs around. The company has also just launched the Internet-based e-Omnitel.)

‘Little difference was found between online survey respondents and telephone survey respondents,’ reports Thompson Lightstone’s Cliff Swaters, director of Omnitel (see graph above). But what differences there were turned out to be revealing.

For instance, online survey participants, who naturally tended to be more familiar with the Internet, were more comfortable about sharing their beliefs online (at 37%), than telephone respondents (19%). And demographically, the online survey did skew somewhat younger, with 48% of online respondents aged 18 to 34, compared to 33% for the telephone sample.

However, research suppliers conducting Internet surveys can overcome such demographic differences by weighting the data, and establishing controls like quotas. So for instance, once you have the required number of respondents from the 18-to-34 age group, or the required number of women, no further surveys from those categories would be included.

Rob Shields, VP customer relationship, marketing and loyalty at The Bay, has a simple but effective control when overseeing the department store’s in-house online research.

‘Sometimes, respondents will rush through the survey just for the points,’ he admits (The Bay offers HBC reward points as an incentive to fill out the questionnaire), but if he notes a survey was completed in just one minute or only ones were clicked all the way down the form, he just throws it out.

As far as leaving out 40% of the population goes, Levy’s experience is that, ‘in most cases – though not all – results of online testing result in the same business decision being made.’

Getting past the concerns, many research professionals say the Internet has its benefits and place within the wide-ranging tool set used to gather data.

Gadd likes the fact that Internet respondents are ‘more candid and thoughtful.’ For someone who describes his company’s specialty as ‘show-and-tell research,’ Gadd also particularly likes the visual capability of the Internet. He can show respondents a picture of a new product to gauge reaction, or test an ‘animatic’ commercial, a less costly, cartoon-like version of a trial commercial. (See ‘New model for a new medium’ on page 25 for more on exploiting the interactive capabilities of the Web.)

Steve Popiel, senior VP, research and development at Goldfarb Consultants in Toronto, says Internet surveys are very cost-effective as well. His firm even does online focus groups with online moderators. While Goldfarb itself does not have a panel, as part of London, Eng.-based international advertising conglomerate WPP it does have access to LiteSpeed, WPP’s non-profit research centre. For internationally focused clients, LiteSpeed also has the capacity to canvas AOL subscribers worldwide.

Despite its convenience and promise though, ‘the Internet paradigm shift in the research industry is still relatively new,’ sums up Market Facts’ Greensmith. ‘It’s all more hype than action right now.’

New model for a new medium

Hotspex lets customers design the products

While traditional market researchers struggle to rein in the Internet badlands to better conduct traditional polling, some companies are parlaying the medium’s natural strengths into whole new research models.

One such company, Toronto’s Hotspex, is exploiting the Web’s visual edge to the max with innovative ‘mass involvement projects.’ The company’s latest project invites any and all consumers to their site,, ‘to design the ultimate Airwalk shoe.’ The shoes are sold exclusively by Sport Chek, part of Calgary-based Canadian sporting goods conglomerate Forzani Group.

Airwalk Canada has committed to use the feedback in the next new shoe model, expected this fall, says Szejack Tan, Hotspex’s VP marketing and communications, one of three twentysomething partners. (The three have no formal training in market research, but Tan says they all share an interest in innovation and technology.)

Consumers are driven to the Web site by posters displayed in 80 Sport Chek stores across Canada, as well as by links from the store’s own site, With a self-styled ‘edgier approach,’ the goal of the graphics-heavy site is to generate fun and get consumers involved.

‘Nobody said learning about customers had to be boring,’ says Tan.

This is high-tech grassroots marketing that gives power to the people, but is it market research or promotion?

Tan says it’s both. While the company can filter out target demos by age, gender and city, its 60,000 plus database is not a controlled sample, and Tan readily admits that ‘you can’t extrapolate the results over a certain population,’ as in standard research. Project results offer a ‘quick read’ only. However, he says there is qualitative value to the work and more importantly, it offers clients ‘the seeds of innovation’ for product development.

‘It acts as the beginning for a good actionable idea.’

Perhaps the greatest value of such hybrid projects is the promotional angle. ‘Why not involve people from the ground floor and make them feel like they are contributing?’ questions Tan. This generates consumer loyalty and a sense of ownership, as well as awareness and anticipation before the new product even hits the shelves.

Next up for Hotspex? Going 3-D.