Analysis and the Internet: Caveat emptor

Colin Tener is president of Tener Solutions Group, a customer relationship management consultancy based in Toronto....

Colin Tener is president of Tener Solutions Group, a customer relationship management consultancy based in Toronto.

One of the more powerful aspects of the Internet is the ability to customize a truly one-to-one message for each prospect or customer who visits your Web site. The technical ability to push back Web content in the form of banner ads or a customized home page has revolutionized the way organizations think about the messages they can send. But if the technological problem is solved, the bigger challenge remains. Exactly what should the message be?

There are a number of software products that are designed to analyze clickstream data and match appropriate Web content to observed interests. These solutions are driven by the Web pages you’ve visited as well as the way you navigate around a site, coupled with whatever personal information you have provided about yourself in the process. By analyzing this information, patterns are identified that indicate what types of messages will be most relevant. And since these messages are more relevant, they lead to repeat visits and higher sales.

At least that’s the story the vendors tell.

As we’ve seen with other so-called software revolutions, however, caveat emptor should prevail. Many of us went through the "neural net" period a few years ago, where vendors essentially told us not to worry our little heads about hidden layers and other complicated matters and just focus on lift. The problem is that marketers actually do want to know why a particular action is recommended.

Unfortunately, many vendors of Internet analytics are reluctant to reveal the engine behind their selections. What’s behind the black box? Is it a predictive model? A neural network? Using what measure of success? If you don’t know what the process is trying to predict, how do you know if you’ve got the right decision rule? Should we be predicting banner ads that are clicked, selections that are placed in the online shopping basket or only those items that are actually purchased? How do we know that the algorithm worked if we don’t match Web-page selections back to transactions with the proper test and control protocols?

Of course, lots of low-hanging-fruit opportunities exist without getting into sophisticated statistical analysis. If someone just purchased a book on Italian cooking from your site, then maybe they would be interested in another one. Or perhaps they might like a book on Italian wine. How about travel books on Italy? And what about information on an online grocer that guarantees next day delivery of fresh, gourmet ingredients? At some point in this sequence we move from the obvious to the not so obvious, at least in terms of likelihood to buy. And at that point, the need for analytical rigour arises.

Faced with potentially thousands of choices of messages to send, we need an analytical process that will identify those with the highest potential for success, whatever that may be. Because that raises another issue altogether. What if the key message we want to send isn’t cross-sell at all? What if we suspect that this customer is planning to switch to a competitive Web site? This situation is no different than our "old economy" banking clients, for example. They want to have the results of predictive algorithms loaded into call centres so that when a customer calls in, the most likely products are displayed along with supporting scripts. But if the customer is showing signs of cancelling either specific products or their entire relationship, then they want that information highlighted to the customer service rep and appropriate dialogue initiated.

Because of this need to both tailor and control the message, I am convinced that black box Internet solutions will have a limited lifespan in the marketplace. Users will want to be able to develop their own algorithms and decision rules, since that is part of their sustainable competitive advantage. If they can’t open it up and see how it works, they will not be able to tailor it to their needs. Sooner or later, they’ll look for a solution that is more open.

Stay tuned. Next month we’ll discuss the Holy Grail: combining real-time clickstream data with offline data captured in the data warehouse.

Colin Tener can be reached at (416) 585-2900 or by e-mail at

The latest in tech: Why it matters and how you can use it

With the pandemic continuing to keep in-person events off the table, innovation isn’t just a bonus – it’s an imperative. ...

With the pandemic continuing to keep in-person events off the table, innovation isn’t just a bonus – it’s an imperative. Companies need creative tech to engage with consumers, whether that’s live-streaming, gaming or data-driven personalization. This is the year that remote comes first, making it an opportune time to experiment with format, devices and tools.