Consider this: Video rental giant Blockbuster records more than 14 million transactions a week. There are 21 million Visa cards in circulation in Canada right now. And Wal-Mart Canada processes 100,000 transactions per day from 5,000 vendors.

Virtually everyone today collects and stores transactional data. But is the potential of data being properly exploited? Can it really help marketers get insights into their customers? Is the technology now mature enough to fully reap the benefits of the data that’s being collected?

Eric Apps, president of Angoss Software, a Toronto-based developer and vendor of data mining solutions, says the answer is yes – and no.

In general, he says, the technology has improved and client awareness of how it can be used has increased exponentially. Companies have moved beyond their preoccupation with the Y2K issue and building a database infrastructure, says Apps, to embrace data mining as a key competitive weapon.

At the same time, he says, the technology is being continuously upgraded to ensure better performance, ease of use and broader deployment. And data mining these days is becoming more integrated with the Internet, as companies such as Angoss, Oracle, SAS and others make their data mining offerings Web-enabled, allowing data delivery via either Internet or intranet architecture.

‘Data mining revolves around achieving deeper understanding of your customer base – segments, profitability, risk and the like – and better predictors of their likely future actions and behaviours,’ says Apps. ‘Data mining technology – both directly and through integration with other solutions – is becoming more efficient at revealing these patterns and relationships in useful and actionable ways.’

But Apps says it will take further ‘drilling down’ and more refined applications if companies are to reach the elusive market of one.

In the meantime, however, there are plenty of reasons to think progress has been made.

Barb Cromb, vice-president of TD Access, the Toronto-Dominion Bank’s interactive banking division, says data mining tools have definitely improved over the last few years.

‘Our results – for example, who will be likely to acquire a new credit card – have been bang-on,’ she says.

Earlier this summer, TD Bank invested in SAS Institute’s Enterprise Miner data mining software solution in order to better analyze and understand the value of its customer data.

‘It’s becoming more critical to us as we move to a sales mode that is not so wholly branch-centred,’ says Cromb.

‘Understanding customer behaviour and quickly responding to customer needs are keys to long-term success in an increasingly competitive and global financial services sector,’ she says.

Which tends to be a whole lot easier, say the experts, when you have the data to begin with.

According to Apps, the single greatest obstacle to the evolution of data mining in this country is the paucity of usable data. Outside of the telecommunications and financial services categories, he says, most Canadian companies don’t have the same depth and dimension to their data as their U.S. counterparts.

‘So the first thing many of them will find is that their organizational data assets are inadequate,’ he says. ‘That’s depressing, but useful as a starting point to improve. As they move forward to start capturing, exploring and analyzing their data, good data mining technology [can] yield patterns, relationships and anomalies that contradict the perceived wisdom, or that are completely unexpected.’

That’s exactly what Jim Carswell, general manager of credit scoring at the Canadian Imperial Bank of Commerce (CIBC), discovered. He says data mining software helped reverse a typically held view of customers who were likely to default on their mortgage payments.

‘We had felt that people who had occasionally gone past-due with their mortgage – say four or five days – two or three times were at risk of defaulting down the line somewhere,’ explains Carswell. ‘It turns out they were just people who were a little late.

‘Now, look at people who were never late and suddenly they miss a mortgage payment. All of a sudden, there was no money for the mortgage. There has been some shock to their life – job loss, or a sudden, unanticipated expense.

‘Data mining allowed us to uncover relationships within the data we may not have known before. We could have done that with some of the purely statistical methods, but today, data mining tools can allow an adequately-trained business manager to do it instead.’

Adequately trained like Carswell?

‘I’ve never taken a statistics and probability course in my life,’ he laughs.

Also in this report:

– Web-based data mining turns up gold: Virtual explosion of available data supplements internal database systems p.D14