How Microsoft Access can damage your data quality

OK, so I've bitten the bullet. Much against my better judgement I'm making a concerted effort to learn Microsoft Access 2007, to allow me to produce some data tables with Unicode characters.

Plugging away, I found an interesting aspect of field input masks which is guaranteed to produce data quality issues. When adding a field mask to Access 2007, it "helpfully" provides a number of ready-made options:



Having a UK edition of the software, Microsoft have helpfully provided a telephone number mask and a postal code mask that it thinks covers the UK. Looking at the postal code mask itself:



and we see how it is made up. '0' indicates a required digit, 'L' a required letter, and '>' forces upper case.

Now, in the UK, AA99 9AA is indeed a valid postal code format. It is, however, only one of seven valid formats:

A9 9AA
A99 9AA
A9A 9AA
AA9 9AA
AA99 9AA
AA9A 9AA
AAA 9AA

Thus, whilst this mask can take OX19 6RY, it can't take OX9 6RY or SW1A 4WW or S1 1AA or N45 1AP or .... well, most of them.

Microsoft may feel that they are being helpful by adding this sample mask, but we all know that programmers, like most of us, will take any route that make their life easy, and are unlikely to make any attempt to alter this input mask to make it valid for the UK, let alone valid for postal codes in every other country. And we know that this happens - my God, don't we all suffer regularly at the hands of forms designed like this? Many programmers would not even be aware that the mask is not valid for most UK postal codes - they trust that the software provider has done their homework.

Back to the drawing board, please, Microsoft. This is not helping in the fight for better data quality.

Eurotunnelvision

A slight move off topic, if I may. It's that time of the year again when the Eurovision Song Contest takes place, accompanied, as always, by the sounds of weeping, wailing and gnashing of teeth from the losers, who always try to find anything to blame apart from their own performances or the quality of their songs.

One of the most common complaints is the idea of block-voting, accompanied by a rather embarrassing lack of geographical knowledge. This year's Dutch song is doubtless very suitable for, and popular in, the local pubs of The Jordaan in Amsterdam, but is hardly material that will be popular elsewhere in Europe. Inevitably it failed to get through the semi-final. René Froger, one of the Dutch singers, immediately complained that this was due to block voting: "it's so strange that all the Balkan countries are through to the final", he wailed. And worst, no report I've seen pulled him up on it, seeding and strengthening the myth that everybody hates us/loves them.

Grabbing a pencil and a scrap of people, it took me no time at all to work out that six Balkan countries indeed made it to the final. And five did not. Not quite "all", and certainly no evidence of the great conspiracy so many profess to see. He'd have done better to point to the Nordic block, or the Caucasus block, or the Mediterranean block. If anybody should complain, it's the Central Europeans, none of whom got through.

Back to school, René!

For a map of those through, check Wikipedia here

Flying share

Another great example of how ignorance affects data quality from web forms (and can lose customers and money!) has come my way, this time courtesy of the site Flying Share. Flying Share are offering their users in the US, UK, Canada and Australia, via this form, a free USB drive.

All well and good until you get to the "ZIP" field (ZIP? Not outside the USA, good people of Flying Share - you probably mean "Postal code"). And there you find that the field will accept five characters and no more. So anybody in the UK or Canada wanting a free drive must either give up at this point, or provide a truncated code.

As drives are being sent out postally, there will clearly be a huge problem of undeliverable and returned drives. I know that this error has been pointed out to the company concerned, but no action has yet been taken, though correcting it would take seconds.

I do wonder at what point anybody at Flying Share will feel the need to act on this. Perhaps when the costs of returns reaches astronomical proportions? How many (potential) customers might they have lost before then?

I made a blog entry in March about Vietnam, where a postal code exists but without the populace knowing about it.

Two more examples caught my eye this week, in both cases where a country announced the introduction of a postal code system: Nigeria and Dominican Republic.

In both countries a postal code exists - in Nigeria's case since 2000 - and in both cases this information has been placed at one time or another on the national postal authority's website. It is very interesting that in all of the comments placed in reaction to these news items up to this point, only one persons suggested that they thought that a postal code system was already in place.

Regardless of the reasons why the existing postal code systems in either country had not been fully publicised before now, this is another indicator of how careful form designers need to be in their use of required fields. Nigeria may have a postal code system, but if nobody knows about it or uses it, requiring that field on a form will only lead to customer loss and data quality reduction.

For more information about web forms for an international audience, download the free e-book for here.

Free web form e-book

I released today my new book: “Better data quality from your web form - Effective international name and address Internet data collection”. The good news is that it is completely free to download and use, so go to http://www.grcdi.nl/book4.htm to download it. Spread the word!

Web forms are a source of immense and continuous frustration for many people as the forms almost universally fail to take any account of variations in personal name and addressing conventions used throughout the world, so that customers have to struggle to clear the hurdles placed in their way by the forms. Reports suggest that almost 9 out of 10 customers have problems when attempting to carry out online transactions, and the result is a huge loss of income from lost custom paired with the collection of very poor quality data.

This e-book (in Acrobat pdf format), which is free to download and use, attempts to fill the gap left by most works about usability by concentrating on the experience that your international customers have with your form, and how it affects them, their relationship with you and your data quality.

This has been done using many examples and without going into too much depth about the idiosyncrasies of international personal names and addresses, and avoids technical discussions – links to get more information are provided where appropriate in the text.

Any feedback is most welcome.

Enjoy!

New product releases at GRC Database Information

We've been busy in March preparing updates for most of our products. These include:

- an update to the Global Sourcebook for Address Data Management

- a new release for many of our data tables: place name/postal codes, address elements, job titles and others

- a new version of our address parsing, standardisation and formatting software GRCTools.

More information on our website.

Now you have your postal code system, how about telling us about it?

My attention was caught today by an online discussion here hosted by the English-language Vietnam News, about introducing a postal code system to Vietnam to resolve the problem of duplicate thoroughfare names within cities.

Of interest to me is that Vietnam has had a postal code system for years, updating it from a 5-digit code to a 6-digit code in 2004. Clearly, though the system has been designed, it has not been implemented to the extent of informing the residents.

This is not as unusual as you might think. Some countries - Bahrain and Nicaragua spring to mind - are coy about their postal code systems, even to the extent of requesting people not to use them because sorting continues to be done manually. I've had people from Costa Rica swear blind to me that their country has no postal code system, even after I've pointed them towards the postal website describing it. Clearly, whilst some countries are happy to design postal code systems, the expense of implementing them - mechanised sorting systems, information dissemination and so on, tends to put a brake on following the projects through.