BBM is bringing Arbitron’s Portable People Meter (PPM) to Canada, marking the first-ever commercial deployment of this technology in the world. In Canada, panelists and station participants are already being recruited, and BBM hopes to have pager-sized PPM units measuring Montreal’s francophone viewers by late next year.

But there is concern in the Canadian industry that the PPM is not yet ready for prime time, especially given that the U.S. rollout has been put on hold.

Tales of aberration and low compliance rates from the U.S. test in Philadelphia have some agencies, and the Association of Canadian Advertisers (ACA), calling for a Canadian PPM test before establishing it as a third currency in broadcast audience measurement.

The ACA is so concerned about the accuracy of PPMs that the organization’s VP of media and research, Bob Reaume, has undertaken his own investigation into the technology.

Reaume recently travelled to the U.S. to speak with research professionals from the broadcast, association and agency sides of the business about the problems discovered with the Philadelphia test. Based on his research, there are several issues that still need to be addressed before PPM technology gets the OK for deployment in Canada.

What are some of the problems that Nielsen and Arbitron have discovered through the PPM test in Philadelphia?

Reaume: Let’s start with the overriding premise that being in the general vicinity of a TV’s audio signal is the same as watching television. It’s obviously not, and in fact, this measurement technique represents a step backwards, not a progressive step forward in TV audience research. We should be moving from measuring vehicle distribution or exposure to advertising exposure or attentiveness.

One of the obvious problems with this technology is the tendency for incidental or accidental ‘viewing’ to be captured, resulting in substantial unintended ‘viewing’ from out-of-room. This accumulates from two different kinds of out-of-room ‘audience’ – outside the room but nearby for a protracted period of time, and transient behaviours [briefly in and out].

And that’s just the unintended viewing in-home. What about all the secondary viewing that is captured outside the home? Can we really give the same value to inattentive audio exposure in a bar or a retail environment? If the technology had the capability to separate in-home and out-of-home, we might at least be able to treat it more like ‘secondary readers’ for magazines.

Then there’s the problem of the device continuing to accept signals when docked through the night. Some of the numbers we have seen confirm that it is counting a sizable number of ‘viewers’ who have fallen asleep. This happens because it continues recording viewing while docked to recharge and download. During the day there is a motion detector to keep it honest, but at night in the docking unit, it is quite still, yet still listening.

In the Philadelphia tests, late night audiences soared to double previous levels, and figures for specialty [U.S. cable] channels in that time block showed an unbelievable nearly five-fold increase. This is a major inherent flaw in the technology. It is actually designed so that when used precisely as intended it credits ‘viewing’ to substantial numbers of persons who are asleep. We can only hope that there will at least be an edit rule for dead people.

What other concerns do you have about PPMs since your visit to the U.S.?

There is also the question of very, very low compliance rates [the percentage of people with meters who use them as instructed]: only 13% in the February numbers and below 10% more recently in the Philadelphia test. That tiny level is simply unacceptable; no further discussion required.

That’s on the TV side. On the radio side, there are other issues. There’s evidence that talk radio signals will be disadvantaged as compared to music stations. The encoded signal apparently carries better on music radio than talk. What do you do about joggers and teenagers and their private headphones? And what about accidental listening that will occur frequently in a second language? This is going to be particularly acute in the dual-language market of Montreal.

The general consensus I found in the U.S. was that this technology is nowhere near ready for market. In our opinion, all of these issues can and should be tested in a Canadian environment before we give it the go-ahead.

BBM says the portable meters are a cost-efficient way to expand measurement in Canada. But if the compliance rates are low, won’t they be spending more money on recruitment efforts?

That is true and not only for recruitment, but for installation and maintenance as well. The Philadelphia test clearly indicated that compliance was an issue. But there are many ways to get compliance up, including increased coaching, moving from a self-install to service-rep install and increased follow-up contact. Of course all of this costs money.

It’s not even clear that the device will ever be able to be used for both radio and TV with the same sample. The prudent course, and frankly, the only proper research protocol in any case, is to put it in trials and study it.

Is there a problem for advertisers if PPMs aren’t designed to measure children?

Yes, we think there’s a problem in this area as well. Substantial amounts of ad spend target this group in English Canada. It may not be particularly valid for Quebec today, where advertising to kids is currently not allowed, but presumably the contemplation is for this technology to eventually be used in other markets.

In the U.S. tests, only kids six years and older were deemed appropriate to wear the receiving device. BBM’s plans for Montreal contemplate including children two to 11 in the sample, our industry’s standard demo for kids. Parents now push buttons or fill in diaries for their kids, but seriously considering having two-year-olds with these units pinned on their diapers, or a four-year-old properly complying, is beyond reason.

What are your recommendations to BBM – and the larger industry – for the implementation of PPMs in Canada?

Here is my wish list for next steps:

1. BBM must obtain independent accreditation of its PPM technology and methodology from the Canadian Advertising Research Foundation (CARF) to ensure that it meets minimum standards for electronic audience research.

2. A specific Canadian study must be undertaken, preferably in two markets – one unilingual and one dual-language.

3. Testing must last at least one year; seasonality is definitely a viewing and listening factor.

4. In-tab compliance levels for the test must be sufficient to allow for comparisons of estimates for individual stations for selected demos.

5. Special studies must be undertaken to determine the reasons for all significant audience differences from the current collection methods.

6. A special study needs to be completed on children’s compliance.

7. An ‘If Go’ Committee with equal representation from broadcasters, agencies and advertisers should assess all studies and make the decision whether to deploy the PPM in Canada or not.

[BBM’s current plan calls for a committee made up entirely of broadcasters that will decide when – not if – the numbers are ‘correct’ and the PPM is ready to go commercially.]

The issue in question: BBM responds

By Ron Bremner

BBM chose to deploy the PPM technology in Canada, not because it lacked an alternative, but because BBM believes personal people meters will lead to significant improvements in audience measurement in Canada. The inherent cost-effectiveness of PPM sets the stage for larger sample sizes. It also offers the potential for single-source electronic measurement of both television and radio.

The alternative to PPM was, of course, Picture Matching, a technology that has proven itself in Canada, and could easily have been selected for use in Quebec without concern. After all, Picture Matching does make a smooth transition from analogue to digital. And it does produce minute-by-minute data that can be translated into commercial ratings if that is what the industry wants. But, on balance, our members concluded that PPM offers the greatest mid- to long-term benefits.

The perception that Nielsen and Arbitron decided to conduct further testing of PPM in Philadelphia because of problems with the PPM technology is inaccurate. There are no problems with the PPM technology, just a desire on the part of both organizations to resolve matters related to definitions and understanding.

Similarly, much has been made of the so-called incidental and accidental viewing captured by PPM. The PPM is calibrated so that the viewer must be within about eight feet of the television – hardly a cause for concern. In our discussions with the industry, it has become good sport to conjure up scenarios where PPM could incorrectly capture viewing. But in reality, the vast majority of media exposure is straightforward and unambiguous. I wish that could be said of some of the current meter systems.

One of the virtues of PPM is that every exposure is treated exactly the same. With button pushing, the viewers have control over when their viewing is counted. But with PPM, if you are present, you are counted. You have no choice in the matter. While it is true that a PPM that is docked overnight could potentially attribute viewing to someone who is sleeping, that is also true with conventional people meters. But with PPM, the system knows the meter is docked and therefore the audience measurement service can develop edit rules to deal with such ambiguous data. Periodically there may be some fuzzy data. But compared to some current systems, we will still be way ahead of the game.

This is an important point. Those who challenge PPM often compare it to some platonic ideal rather than the audience measurement capabilities we have today. If Nirvana is selected as the frame of reference, new approaches will always be perceived to be wanting, and no progress will be made.

PPM is a solid advancement is capturing viewing and listening. Unlike in the U.S., PPM in Canada represents only a minor modification to the system that we already have in place and with which we have considerable experience. That is why we are not the least bit concerned about U.S. compliance rates, for example. Virtually, every aspect of our methodology, from enumeration and recruitment to processing, will be done using the tried and true methods that we currently employ. In this regard, we are indeed fortunate that we do not have to take our cue from the U.S.

That said, one area we will be looking at is the measurement of children. If it becomes apparent that measuring kids 2 to 5 is inappropriate, we will move to a 6+ age break.

Turning to radio, there has been a suggestion that formats like talk radio will be disadvantaged by PPM. This is factually incorrect. The research shows that while code density is greater for music formats than talk radio, there is no differential effect on the audience estimates, because in both cases the code density is dense enough.

Finally with respect to an audit, BBM has always believed in third-party objectivity, and has said repeatedly that Canadian audience measurement services in Canada should be audited. As we consider the evolution of our meter service in Canada, a Media Ratings Council audit of our service is definitely on our agenda. And, as in the past, we will involve all segments of the industry.

BBM’s Board, which consists of agencies, broadcasters and advertisers, supported the introduction of PPM in Canada because it believes that PPM will lead to significant improvements in audience measurement. But to ensure the sustainable growth of meters in Canada, we must engage the whole industry.

Ron Bremner is VP television at BBM Canada. He can be reached at: rbremner@bbm.ca.