IAIDQ Blog carnival for Information/Data Quality, December 2010

It's my pleasure and privilege to again be hosting the IAIDQ blog carnival, this time for posts made in December 2010. I'm glad to say that the festivities didn't reduce the flow of top quality posts, some of which I have highlighted here.

If you work in business and you're attempting to get a data quality initiative off the ground, William Sharp, over at the Data Quality Chronicle, has outlined a data quality program's 10 best practices. A useful checklist whatever your data quality background.

Jim Harris' steady stream of wisdom over at the Obsessive-Compulsive Data Quality Blog hardly let up in December, with a look back at the best data quality posts of 2010 (which I have to mention, because Jim kindly included not one but two of mine in the list!).

I've been thinking (probably rather more than is healthy) quite a lot recently about data silos, and Jim casts his usual sensible eye over just that subject when discussing The Good Data.

I don't know where Jim finds the time to read, but he's read "A Confederacy of Dunces" by John Kennedy Toole and he discusses our propensity to delete data in A Confederacy of Data Defects.

And finally, whilst Jim was looking back over 2011, Steve Sarsfield, in his data governance and data quality insider blog, was looking forward to 2011. The brave Steve has made six data management predictions for 2011. I'm wondering if that's a misprint - won't be be waiting until 2051 for most of those things to happen? ;-) What do you think?

The myth of deliverability.

I am often asked whether an address is "deliverable", and not everybody is happy with my usual response that it depends on the mood of the postal staff on duty that day.

The point is that "deliverability" is unmeasurable, unscientific and has little basis in reality. Address validation software will often give a figure for the number of deliverable addresses within a file, such as 80%, but don't be fooled - this does not mean that, if you send a letter to each address within that file, 80% will get there and 20% will not. These numbers have been created to give some sort of feedback to the user, and to make files comparable with each other when run through the same software processes.

So how come there's no way to measure deliverability?

You could look at a country like The Netherlands, with its neat address system, and boldly state that a deliverable address is one where the postal code and the building number are present. Mind you, if either contain a typo, the mail may be deliverable, but not to the desired recipient. Equally, whilst TNT Post is happy with that much information, because it will get them to a letter box, to make a mailpiece deliverable to a person, more information is required - a sub-building indicator (as many addresses may share the same house number/postal code), a name, department and so on.

So, I've got all that. So the address is deliverable. Right?

A mailpiece containing my correct postal code and house number took 6 months to reach me. Not because the information was wrong, but because a second, stray, postal code had wormed its way into the address block, sending the mailpiece around the system ad infinitum. It was only when a particularly awake postal worker saw and crossed through the stray code that the mailpiece could get sent on its (correct) way.

But not having a full address, or even any address, does not make a mailpiece undeliverable! I remember the TV program That's Life! on the BBC successfully receiving mail sent with just a drawing of a prominent set of teeth on the front - an allusion to Esther Rantzen's somewhat toothy smile.

This Christmas a card was sent with this address on it:

Mr & Mrs T Burlingham?
Near the golf course in Thetford,
Norfolk.
Trevor is a photographer. This might help

It did help. This undeliverable address took just 2 days to arrive. But this isn't just a British thing. How about one of my favourites:



Translation:

Vukasin 6 years and Jelisaveta 3 years
I don't know the family name
PRUSKA GORA
SERBIA
Their father is big, he drives a Citroën Belingo.
He works on the little trains for tourists.
Postman: Please find them!

And he did!

Praise for a form

You know that I have a thing about forms, and I spend a lot of time criticising them. But I did always said that if I found an example of a good form, I'd sing its praises on high.

So, this is me singing.

The form is not an international form - it's for the Dutch market and is in Dutch - but is shows some good practices that I'd like to highlight. The form is from ShopPartners .



Dutch addresses are of the few in the world where a whole address can be derived from very little information - in our case a postal code and a house number - and the form starts by asking for that information to save the customer from typing their whole address - and the less you make your customer type, the happier they are. They show very clearly the number of steps you'll have to go through to order, and the section to the right gives clear examples of what they want should there be any doubt in the mind of the customer.

The second step expands the form with the address autocompleted. Nothing earth shattering there.



The form asks whether the customer is a private individual or a business - a great way of preventing customers from having to fill in or skip over fields which are not relevant to them, such as VAT numbers.

But what did make me smile - no, it made me get up and jig around my office - was that this is the first Dutch form I have come across which does not ask your form of address (Mr or Mrs - no other choices are given) and use it to assume your gender. Somebody has thought about this, and apart from Mr and Mrs the customer can also specify a department name or the name of a family. Still no chance for me to add Dr, Professor, Lord, Sir, Dipl. Ing., Mag. or any other one of the hundreds of forms of address that exist; but I do get the change to choose NONE (my own preference) - an escape route for customers not covered by the options provided - and this is rarely allowed on forms.

And again, descriptive examples of what is required are provided to the right of the fields.

Now, plenty of usability experts will pull me up and mention element placement, colours and numerous other "problems" with this form, but for me it's a huge step forward! Well done ShopPartners!

Want to know more about collecting good data from your web form? Download my free e-book here. And no form to fill in either!

Robert the Carrot

A friend told me an anecdote from the time he was working in a Chinese restaurant. A customer, called Robert, wanted to get a tattoo of his name in Chinese and so asked one of the Chinese staff to write down his nick name - Robbo - as a pictogram.

There's no Western "r" sound in Chinese, and the staff member subtly altered the pronunciation of Robbo to something closer to Lobbo - which means "carrot" in Chinese. The pictogram was duly drawn, and Robbo happily went off to get himself tattooed large as being "Carrot".

This resonated with my data quality genes in two ways. The first is what I call the Chinese Whisper effect. You know that game - one person whispers a word or phrase into the next's ear, and so on down the line, and what comes out at the end is often completely at odds with how it started. Data quality is like that - at every interface between information and data and between data systems, the quality of data goes a little more awry.

The second has to do with ignorance. Most organisations think their data is great simply because they don't understand it, and that's especially the case with names and addresses. If you can't see the problem, or, indeed, see that there is a problem, you can't correct it.

Robbo lives in ignorance about his tattoo and is probably still mighty proud about it. He may get a few sniggers if he ever goes to China, but that's about it. Unfortunately, data quality issues arising from processes which work like this can be much more dangerous.

Today we welcome ....

.... five new countries and territories, and we say goodbye to one. The net number of countries and territories in the world without a postal code system increases by four.

Did you know? Had you noticed? When I asked two days ago at a speech at Post*Expo in Copenhagen, none of the 50 or so participants admitted to having any clue about it.

Today we're waving goodbye to The Netherlands Antilles. We're welcoming Curaçao and Sint Maarten as largely independent territories within the Kingdom of The Netherlands (as Aruba now is); and Bonaire, Sint Eustatius and Saba as special municipalities within The Netherlands.

How many weeks, months, years or decades will it take before organisations reflect these changes within their databases, processes and customer-facing systems? After all, Saint Martin and Saint Barthélemy (2007) are still largely unknown and unused; Serbia and Montenegro are still far too often lumped together, though they split in 2006; and some organisations still have Yugoslavia as a country, though that died a death in 2003.

Manage your own country list

Twice in the past couple of weeks, when pointing at errors in country lists, organisations have let me know that they will be "looking for a new source" for that list. Far too many organisations use incomplete and unsuitable lists provided by organisations such as the World Bank, United Nations or the ISO. These organisations have their own reasons and imperatives for creating and maintaining lists, which will not be the same as yours, and they must adjust lists to political pressures which rarely reflect reality on the ground.

If you need to keep your country list up to date, and you do, then manage your own. Any country or territory which has a de facto existence needs to be on your list. Though Guadeloupe is part of France, it's geographical location means that it needs to be listed separately to ensure correct address management. Saba may become part of The Netherlands but it won't use the same postal code system or, indeed the same currency. Kosovo must be on your lists because of the linguistic, cultural and addressing differences, regardless of how you stand on its relationship to Serbia.

If you can't rely on the list you're using now, use your own. I'm looking forward to Sint Eustatius and its new neighbours appearing on your website form dropdown very soon.

Avis - not trying hard enough

Before you clock off and look elsewhere, this is not going to be a rant about how bad Avis customer service is - it's an illustration of how broken processes (in this case of information exchange and actions related to that information) can affect a company's bottom line.

I was in Croatia in September and booked a car from Avis for a trip into the mountains. I chose Avis because their location was close to where I was staying. Come the day of the collection, we went to the address given (at the Sheraton Hotel) but couldn't find Avis. At the reception we asked and they informed us that Avis had moved to a location a few kilometres away on 1st July. They were turning away "tens" of customers every day, who were all coming to search for Avis.

It took us a while to get to the new location, and when I vented my frustration to the staff there they just shrugged their shoulders. "We've e-mailed head office - what more can we do?". They weren't bothered about the "tens" of customers they were losing.

Back in The Netherlands I searched the global Avis website for a complaint form. No, they don't have that - they are clearly confident of the quality of their services - so I sent "feedback". After some days the reply came back that complaints had to be dealt with in the country of my residence, The Netherlands, even though they have no responsibility for the website. After some weeks I called Avis in The Netherlands (on a premium rate number - this is me trying to help this company to get back its customers by spending my own money) and found that the complaint was being processed - by sending it on to the office in Croatia. Even if Croatia did nothing (and what could they do except send another e-mail to head office?) the Dutch office would then pass this information on to me, and nothing would have actually happened except that the time of a lot of employees and a customer had been wasted. When I explained the problem to the customer service representative at the Dutch office, she admitted it took many months to get any changes made to the website at all.

In the meantime the website is still sending people to the wrong place. If we reckon on Avis losing 10 customers per day because of this I'm reckoning that, as of today, that's 900 customers.

Avis seems to think it's big enough and profitable enough to carry this sort of loss. A slight tweak in one of its processes (which would cost nothing - in fact, it would save money because staff wouldn't have to field complaints like mine) would have a huge effect on its bottom line.

Time to try harder, Avis?

The dangers of obscenity tagging ...

I was pulled up sharp by this report about an XBox user whose account was suspended because he used "offensive language" in his details, the offence being that he lived in a city in the USA called Fort Gay.

I'm continually astonished by the stilted sensibilities of some Anglo-Saxon communities (I have never come across policies banning "offensive language" in systems for any other language) - like most of my fellow Europeans I can't imagine why the word "gay" should be deemed offensive. But this incident highlights the problematic use of obscenity lists which cause problems all the time for large numbers of innocent Internet users.

Gay, for example, is a common given and family name in the English-speaking world, and there are many streets and places named after those people. There are many people with the surnames Duck or Mouse, and some are called Donald or Michael; and they have a constant battle to achieve anything online. The good burgers of Scunthorpe in England have a very trying time with web sites and spam filters (think about it); and imagine the issues that the inhabitants of Dildo in Newfoundland, Condom in France and Fucking in Austria have.

If you do feel the need to check for obscenities, it needs to be done in a more knowledgeable, culturally-aware way than is currently the case.