Games & scripts – stopping at red lights

I’ve been reading about game development lately. It’s fascinating. I’m sure there is much that conventional developers and testers could learn from the games industry. Designers need to know their users, how they think and what they expect. That’s obvious, but we’ve often only paid lip service to these matters. The best game designers have thought much more seriously and deeply about the users than most software developers.

Game designers have to know what their customers want from games. They need to understand why some approaches work and others fail. Developing the sort of games that players love is expensive. If the designers get it wrong then the financial consequences are ruinous. Inevitably the more thoughtful designers stray over into anthropology.

That is a subject I want to write about in more depth, but in the meantime this is a short essay I wrote in slightly amended form for Eurostar’s Test Huddle prompted by a blog by Chris Bateman the game designer. Bateman was talking about the nature of play and games, and an example he used made me think about how testing has traditionally been conducted.Midtown Madness

“Ramon Romero of Microsoft’s Games User Research showed footage of various random people playing games for the first time. I was particular touched by the middle aged man who drove around in Midtown Madness as if he was playing a driving simulator. ‘This is a great game’, he said, as he stopped at a red light and waited for it to change.”

That’s ridiculous isn’t it? Treating a maniacal, race game as a driving simulator? Maybe, but I’m not sure. That user was enjoying himself, playing the game entirely in line with his expectations of what the game should be doing.

The story reminded me of testers who embark on their testing armed with beliefs that are just as wildly misplaced about what the game, sorry, the application should be doing. They might have exhaustive scripts generated from requirements documents that tell them exactly what the expected behaviour should be, and they could be dangerously deluded.

Most of my experience has been with financial applications. They have been highly complicated, and the great concern was always about the unexpected behaviour, the hidden functionality that could allow users to steal money or screw up the data.

Focusing on the documented requirements, happy paths and the expected errors is tackling an application like that Midtown Madness player; driving carefully around the city, stopping at red lights and scrupulously obeying the law. Then, when the application is released the real users rampage around discovering what it can really do.

Was that cheerfully naïve Midtown Madness player really ridiculous? He was just having fun his way. He wasn’t paid a good salary to play the game. The ones who are truly ridiculous are the testers who are paid to find out what applications can do and naively think they can do so by sticking to their scripts. Perhaps test scripts are rather like traffic regulations. Both say what someone thinks should be going on, but how much does that actually tell you about what is really happening out there, on the streets, in the wild where the real users play?

Advertisements

4 thoughts on “Games & scripts – stopping at red lights

  1. Hi James,
    Good blog,again. It reminded me of my time in an online gambling company. What you said about designers having to know what their customers want is absolutely true. It comes down to not just the designers but the product managers whose job it is to know just that.
    In that context UI was very important as well to streamline the website and applications to ensure customers could start playing (and spending money quickly).

    It also reminded me of a short twitter exchange with Michael Bolton a few days ago who complained about itunes. I was tempted to say ‘you’re doing it all wrong, it’s useless as a music player but a good tool to batch convert file formats!’ So yes, users have different perceptions of what an application should do for them.

    • Thanks Thomas.

      I’m not sure Michael would have considered “you’re doing it all wrong” very helpful!

      On a similar note, Flickr has its origins in computer games. It was originally just a chat room for gamers, set up by Ludicorp to promote a game. Users could share screenshots of them playing the game. That ability to share photos proved incredibly popular and the original purpose of the site was lost. The rest is history.

  2. James,

    I really like this post and the recent one that Michael Bolton enthusiastically praised recently on Twitter while he was urging more people to familiarize themselves with your thoughts.

    I am a fan of testing in which testers are encouraged to break free from the happy path and maximize variation throughout many of their testing activities. One approach that I’ve seen work well at a few companies I work with is to include specific instructions designed to achieve that goal such as “in this scenario, imagine you are a user who [is using the application in a highly-non traditional way / e.g., in this case, they have preconceived notions that he should obey traffic rules], and try to uncover possible problems with the following path through the system while being sure to perform actions x and, y, and z …” For the reasons you point out, I prefer that approach (which grants testers more leeway) to the more common style of: “confirm that when you do x, y, z (in this highly-prescribed way), confirm that….”

  3. Thanks Justin. It wasn’t working as a developer, or even a tester that shaped my views on this. It was my stint as an auditor for a big insurance company. Insurers churn huge sums of money around. Fraudulent claims by customers was always a big problem, but so were systematic frauds by employees. An employee who’d gone over to the dark side could skim off life changing amounts of money over a few years that would be barely noticeable in the results for a local office, never mind the corporation as a whole.

    When we audited applications we never looked at the original requirements. We tried to work out where the sensitive parts of the system were and then tried to force the wrong sort of behaviour. Basically we tried to adopt the mindset of crooks, which comes quite easily to good auditors! I had a far more naive approach when I started off as a developer. When I had another stint in development someone said she could always spot the parts of a system I’d designed because the design reflected a rather bleak worldview. Things will go wrong. People will make mistakes, they will be irresponsible or even criminal.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s