O2 website usability: testing, secrets and answers

I left off my previous blog about the usability of the O2 site’s registration process when I’d got the validation code, which was texted to my phone, and I was about to move on to the registration form itself.

This form is a gem if you’re wanting to look at poor usability and sloppy testing, but if you’re an O2 customer then it’s a charmless mess.

  • It applies inappropriate validation, and compounds the problem with an inaccurate error message.
  • It seems too reliant on server side validation.
  • Valid input is cleared if an error is detected in another field.
  • It has too many text input boxes, and too many of them are mandatory.
  • The form as a whole, and the process it supports, don’t seem to have been thought through.

I’m going to deal with only the first issue in this post because it’s not a straightforward matter of usability. It highlights the security weakness of secret questions and answers.

click on image to view

Do companies allow coders to write error messages for users?

I entered the six digit validation code and all the other details, including choosing a user name and password.

I had to choose a security question from the usual set, i.e. mother’s maiden name, first school etc. I chose my mother’s maiden name, then entered the name, “O’Neill”. By the way, that is neither the question I chose, nor the value I entered, but it serves my point.

I got the following error message.

“Security answer must contain letters and numbers and be 1-50 characters long”

That seemed a bit odd, but I stuck a couple of integers on the end of the name.

It then became clear that the validation code and passwords (initial entry and confirmation) had been removed from the refreshed screen that had come back with the error message. It wasn’t immediately obvious that the verification code had been removed because I was below the fold and the message was out of sight.

So I re-entered all the details and tried to submit again. It still didn’t accept my mother’s name, and the validation code and passwords had gone again.

It then dawned on me that when they said that the answer must have letters and numbers they didn’t actually mean that. Maybe they meant that it couldn’t have special characters? So I tried removing the apostrophe from O’Neill.

Yes, that was the problem! They’d created a freeform text input field with validation to stop special characters being entered, and the error message doesn’t actually say so. Oh dear!

However, I didn’t have the chance to enjoy my success. Now the user name I’d chosen was flagged up as being unavailable. And yes, the passwords and validation codes had been wiped.

I re-entered everything and tried another user name. No joy, and of course I’d lost my data again. Next time I just asked them to select a user name for me.

Success at last! I’d registered.

Filling in the form had taken far longer than it needed to, and had left me exasperated because O2 had ignored some basic usability rules.

The validation didn’t make sense for the input that was requested. If you’re asking for freeform text then you should allow for special characters. If you do decide that they are unacceptable then you should make that clear before users input their data, and you should ensure that your error messages are also clear on the point.

Why ban special characters?

The only special characters on my keyboard that were acceptable were hyphens, commas and full stops.

I wish I were more technical and could identify with confidence what O2 were doing, but it looks suspiciously like a very clumsy defence against SQL injection attacks. As far as I know it’s not necessary to ban special characters from free-form text input fields. Programmers should be sanitising the input to deal with potential attacks, or using bound database variables so user input is strictly segregated from the executable code. They should shouldn’t they? Help me out here!

Anyway, even if it is a reasonable precaution (which I doubt) to ban special characters that could result in user input being treated as executable code, surely it should just be the dangerous characters that should be banned?

Beware of not very secret secrets!

Secret questions and answers are a notorious security weakness. They can be ridiculously easy to guess, especially if you know the customer. For a quick introduction to the subject, check out this recent paper from the University of Cambridge Computer Laboratory.

Some people choose to use special characters in their secret answers to make them harder to crack. It hardly makes sense to stop them.

If you are going to use them you should really allow the users to choose their own question and answer. If you really must insist on giving the user no choice then don’t use the same old obvious ones that O2 have.

  • Mother’s maiden name
  • Name of first school attended
  • Name of your pet
  • Favourite sports team
  • Favourite animal
  • Place of birth

That’s a dreadful set of questions. Any number of people outside my immediate family would either know the answer to most of these, or be able to take an informed guess.

To make it even worse O2 have suggested you set up a user name using “the name of a favourite pet, footie team, house, school, street or town in combination with a number such as your date of birth, house number or mobile number.

That’s enough for now

The poor error message, the dubious validation and the rather naïve use of secret questions all combine to give a poor impression of the site. If your approach to testing is based on deriving scripts from requirements, then I doubt if you’ll detect such problems. Rather, you may see them, but you won’t see them as being problems. Even if you do think that they are a problem it might be difficult to persuade the developers.

I’ll return in a day or so to discuss this further in “O2 website usability: beating the user up”, and talk about how testers can and should try to prevent these problems occurring, rather than just complaining about them when it’s too late to make a difference. Basically, it’s a matter of being able to ask awkward questions at the right time!


5 thoughts on “O2 website usability: testing, secrets and answers

  1. Excellent posting. Reminds me of a situation were they also prevented to use special chars because the font map on the Database was too old and changing it because the new build system needed it was too expensive as the input of that system was transported over a chain of other databases which also were “old”.

    At least behind using special chars there might be other reasons why it is not implemented. You are right that information in messages provided must be clear and unambiguous. Certainly this kind of behaviour should trigger us to be aware and test against it.

  2. Another great post.

    Error messages or tech. support articles that phrase things in a positive voice all the time are very frustrating. Sometimes, for marketing reasons, companies decide that they don’t want anything to sound negative or suggesting that a customer has done something wrong. To my mind this causes far more problems and irritation because the message is unclear.

  3. Thanks for the comments Jeroen & Stephen.

    Coders really should not be allowed to write error messages. They can stick in messages as place-holders, but the final content of the messages should be decided by whoever is writing the content for the site. There is a tendency for coders to be slightly impatient with users who don’t get it. That certainly applied to me in my coding whizzkid days. “Give ’em a slap and they’ll know better next time”. I exaggerate only slightly!

    The whole topic of validation and error messages is bigger, more complex and more subtle than being just a matter of coders stopping users fouling up the application with bad input. I’ll be returning to this in my next blog.

    • Indeed Caroline. Is there a tendency for people to think, “oh, it’s just a form – that’s the easy bit”? Are forms pretty much ignored till they are lashed together in a rush, with too many of the decisions being taken by technical people?

      From my perspective, having had a straightforward development background, I think I can see exactly how that sort of mess comes about. Attention is given to the obviously difficult technical stuff. Usability and forms might be difficult, but that’s often not recognised by the technical people, so the problems aren’t acknowledged till it’s too late to sort them easily.

      I wrote the blog last April. I’ve just checked, and the form hasn’t changed. It’s still got the same terrible features. I suspect that there are some good people in O2 who look at it and weep.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s