“Who you gonna believe: me, or your own eyes?” The famous quote from Groucho Marx illustrates a central point that I continue to make with students and clients: to ensure a complete understanding of your marketing database, you must routinely browse your customer data files. It’s an old fashioned approach, but necessary when you consider that bad data hygiene is the leading cause of CRM failure (see Focus Research Ten Drivers of CRM Failure).
I was reminded of this when I attended a database marketing seminar, and the topic turned to sophisticated, expensive data mining software. After some discussion, it was generally agreed that simply browsing customer data – without using advanced statistical approaches – had as much or more “data mining” value for marketers. While the Real Estate industry emphasizes location, location, location, in data mining, it is patterns, patterns, patterns. Here are some of the patterns that can be revealed with what I call “Eyeball Data Mining.”
- Faulty data hygiene patterns: These include blank fields, inaccuracies, duplicate records, incomplete data entry, missing data, outdated information. By observation, a pattern may reveal itself that contributes a great share of problems. One of the most common examples is Order entry: many operators who take phone orders may routinely key the same default source code for all orders, instead of finding the correct code to apply for each specific order.
- Database conversion patterns: when a new software program is implemented, a Quality Control process for the data conversion is often overlooked. Users would rather pay attention to the software bells and whistles. Old codes are not mapped properly to new fields. Old dates are converted badly. Contact names get jumbled with company names – you get the idea (also known as “GIGO” – garbage in, garbage out).
- Missing data: perhaps the trickiest data pattern to see is one that is not there: when the data you would expect to be database-resident is missing completely. As marketing becomes more multi-channel, the data exchanges between each platform become more challenging. Here are a few patterns to recognize:
- Customer number ranges. Gaps in your customer numbering could mean you are missing an entire set of records from a period of time.
- Inconsistency between reporting systems. Experienced marketers know that you can’t expect your marketing reports to tie out exactly with accounting reports. For instance, there are always differences in order cutoff dates versus paid dates. However, you should look for large differences in customer or transaction counts, which may indicate serious problems with your data handoffs.
- Make certain that the results from your ETL (Export, Transfer, Load) utilities are checked periodically – ideally by a 3rd party independent of the process itself.
Professor’s Corner:
I Can See Clearly Now – The Advent of Affordable Visualization Software
Data patterns (discussed above) can be revealed with simple, low-tech sorting and filtering tools. These tools are useful for even a novice marketer, yet powerful since they can link to large SQL customer databases. Microsoft Access happens to be my favorite – many users prefer Filemaker as well.
Two quick examples:
- Filtering can detect discrepancies in customer counts by customer type.
- Sorting can reveal gaps in customer numbering and order numbering.
Eyeball Data Mining has its limitations, of course, particularly as data sets grow large and complex. Until recently, visualization software was not accessible to the average marketer, from either a budget or a technology standpoint. Happily, that has changed, and we can report on some tools that you may find helpful in seeing patterns of both bad (and good) data hygiene.
Tableau Software. If one picture is (still) worth a thousand words, than Tableau might be a good choice. Our sources report that Tableau is inexpensive ($2K), powerful and easy to use.
BatchGEO. We used this tool for a geo-mapping project for a client, to show the location of best customers. Very easy to use, you just copy/paste a zip code spreadsheet from your desktop into BatchGEO’s online interface. Then watch a map appear, with the distribution of locations from your spreadsheet. The subscription fee is $99 per month, which you can terminate at any time.
Takeaway: Visualization software won’t necessarily reveal discrepancies by itself, but will raise your curiosity about customer patterns that don’t pass the smell test. It is up to you to take the next step and diagnose the root cause.
