User Interfaces in Games

Games UI Series

For some time I have written about both my professional and social interests on this blog; covering user experience and gaming, but I want to combine them and look at user interface design in games. I think this is an oft-neglected part of games, especially with the usual budget and time constraints, however as with any software design the usability of the user interface can have a profound effect on the user’s experience.

World of Warcraft screen with massive campaign detail

An advanced user experience on World of Warcraft

Usability in games is not restricted to on screen interactions, there is a such diversity of ways to interact with your gaming platform of choice; be it joypad, keyboard, touch screen, or no controller at all. This makes the platform and method of interaction a key part of the user experience in games, as such I will explore the strengths and weaknesses of these human-computer interfaces.

Some games designers and developers think that creating games is completely different to creating other software, because they are creating entertainment rather than tools. However recently as we have seen an increasing overlap between games and applications e.g. Epic Win we can see that these lines are far more blurred than previously considered. Software development has only recently realised the commercial value of user experience, but games developers often consider themselves the audience as well as the creators, failing to realise that their familiarity with their game hampers their ability to see their product impartially; perhaps more frustrated by the focus groups that require them to “dumb down” games than they are in the issues that may cause that confusion in the first place. While games do need to offer challenges in order to evoke a sense of achievement, these challenges should be designed and deliberate and not a hurdle of a poorly designed interface.

I was delighted to see that Edge has added to its staff Graham McAllister; the CEO of Vertical Slice, the UK’s first usability testing company to focus solely on games. This recognition of the need for usability in an industry leading publication can only help raise the profile of the value of understanding your users.

I’m hoping to write a series of game reviews, which look specifically at the UI and give a heuristic review on their strengths and weaknesses as well as offering possible alternative solutions where appropriate. Read More

My fortnight affair with Nokia – a review of the N8

Nokia N8 handset

For the last two weeks, my HTC Desire has been off for repair, and Nokia very kindly sent me an N8 to try for two weeks, so I wasn’t stuck with the terrible handset T-Mobile gave me. The only condition…. this review.

So here it is the highs and lows of my first Nokia handset, from a user experience designer’s perspective.

I was impressed at first, the N8 offers some haptic feedback, which I think is very valuable when dealing with touchscreen interfaces, but in my opinion it’s too indiscriminate, as you get feedback for any action you perform, including scrolling. However the button feedback is very subtle and effective, with a down and up feedback, much better than other touch screen devices I’ve used.

[more]

The handset itself ok, about the same size and weight as my HTC Desire, complete with audio jack, USB, camera and MicroSD expansion in a sealed unit like an iPhone. In addition it has a HDMI output, which makes more sense when you look at the camera on this handset.

The camera is pretty outstanding, it’s 12 megapixels and has a Carl Zeiss lense and includes a flash. The handset has a dedicated photo button, making it very easy to switch into camera mode. Although the case doesn’t offer a way to protect the lense cover, which is a shame, but would add bulk. The camera is so good that if that features heavily into your phone choice you should definitely consider this handset.

The problem for me came with the operating system, it does do a bunch of cool stuff, but Symbian is still feature rich at the cost of usability. Holding down call, allows you to open apps by voice, but you have to know what to call it, e.g. Internet got me nowhere, while it recognised Contacts and Calendar.

The biggest failing for me was the difficulty with which to get to applications. The desktop space, has multiple screens and is easy to customise, but you can only add widgets, not shortcuts to applications; an option called “shortcuts” just offering four pre-populated shortcuts and if these could be customised I couldn’t work out how. Instead accessing the apps requires I press the home button and navigate through a screen that looks like a slightly improved version of the old ‘mystery meat’ Symbian menu, to an application menu, where all the good stuff sits. It’s only saving grace is that it highlights which applications are open with a little green ‘o’ next to the icon in this menu, and that it is easier to close applications than in either IOS or Android.

It seems fair at this point to talk about the application ecosystem, Nokia has its own ecosystem called Ovi, in the same way Android has Google and the iPhone has iTunes and Mobile Me. You can buy apps and backup your contacts in the same way you can on any smartphone . The problem here is that with any smartphone the apps have a really important part to play. The usual suspects do make an appearance, so you can still play Angry Birds on your N8, but there will be less applications available for general consumption. The Ovi store works smoothly and is not unlike using Android’s marketplace, so it works well, but doesn’t offer anything new. A shame as I think the app stores are the areas that now really need reconsidering in terms of usability.

I did get frustrated when I couldn’t find the handset MAC address, so was unable to add it to my wireless network. Information like this can be well hidden, but it should be available somewhere, after a google I found out you need to put a code into the call screen, which for me is beyond obtuse.

The compact charger is a delight, much lighter and smaller than most, although why it isn’t a USB charger I don’t know.

Overall I did struggle with this handset a bit, but this may be largely because I’m already heavily invested into the Google ecosystem, so once I sync my phone with my Google account everything else comes to life. But I have no investment in Ovi and despite creating an account for this review, it’s unlikely I would ever use it in the same way – seamlessly across multiple platforms. I think that the hardware is pretty good, the camera especially is outstanding, but it needs an operating system that has been built from the ground up as a smartphone OS and Symbian still fails on that front.

Thanks to the great guys at WOMWorld Nokia for the loan of the phone, and credit should be given to Nokia for supporting a venture that engages with social media in a smart way. I expect this will be the first and last review I ever get to do thanks to Twitter, but it has been a great experience.

Android and HTC: The mobile power couple

I recently acquired a HTC Desire running Android 2.1 and it has significantly changed the way I use my mobile phone. The primary reason for this can easily be attributed to Android. I knew when selecting a new phone that the apps would make or break it.

So why not go with iPhone? Because HTC make the best hardware on the market.

HTC Desire

[more]

Hardware vs Software

I have been a fan of HTC handsets for several years now, as they regularly push the hardware limits of mobile devices, squeezing in just that bit more functionality than their competitors; however more often than not the software was a letdown, not living up to the potential of the hardware, even in things as crucial using it as a phone.

Apple blew the market away; the hardware is not as groundbreaking, but it made sure that the OS fully supported the hardware functionality, concentrating on the usability of the product in a market that was saturated with terrible UI design.

Windows Mobile

A few months later Android launched.

Apps

The reason my mobile usage has changed is largely because of Marketplace. My old HTC Diamond, has amazing hardware, but runs Windows Mobile and the apps are sold in a more traditional software business model (as you might expect on a Microsoft platform) with a price tag to match. Apple created the AppStore, as a single point for all apps, accepting micropayments through iTunes.

Android has easy access via their Marketplace and also utilises a lot of the other successful Google applications like Gmail and Google Calendar, meaning for me it is taking the tools I’m already using and seamlessly putting them in my pocket. I could check my Gmail on my old Diamond, but it was pull not push and didn’t support HTML; now it is easier to use my phone than it is my PC.

Apps are purchased through Google Checkout with a 24 hour refund policy, so rather than enforcing heavy restrictions on the apps available as with iTunes, users can try and review apps without cost for 24 hours.

Widgets

Android has a desktop like space which you can customise with widgets, which means you don’t need to launch applications to get to useful information, such as your appointments, photos or if your train is on time.

HTC and Android

HTC have a history of taking off the rough edges of the OS interface with HTC Sense. This made my old HTC Diamond very usable and it hid a myriad of Microsoft’s sins behind smooth animation and gradient interface menus. With Android this work isn’t needed, allowing them to concentrate of creating some great looking apps and widgets; that means your phone is ready to run straight from the box, no need to go hunting through the Marketplace for Facebook or Twitter apps, just sync and go.

Development

Fragmentation has always made mobile development difficult and while neither platform is particularly easy to develop for, Apple have recently taken the step of excluding apps that have been compiled from Flash, which could reduce the number of developers with the necessary skills to create apps. Android is Open Source and Java based as opposed to Apple’s Objective C, and there is an increasing market of tools to help compile for your desired OS. Apple no longer has the largest market share, although the expected release in June of a new handset will most likely boost iPhone sales again.

The cost of submitting an app is considerably cheaper for Marketplace than it is any other platform, and the volume of apps is increasing as the Android adoption grows.

Handsets and Networks

Apple has at last allowed networks other than O2 sell the iPhone, but the hardware doesn’t change until Apple release new version. Android is available on many different handsets, so you can pick one that best suits your specific needs. HTC are constantly developing new handsets so you can always get the most cutting edge technology on the market.

Pivot – interesting data navigation from Microsoft Labs.

Pivot is a tool for browsing large collections of data, such as Wikipedia, and really shows the value of a good API and semantic data.

It contains a filtering system on the left to help you dig into and out of the data, combined with a smooth zoom interface as we’ve seen implemented before in Seadragon and Photosynth and some very familiar browser elements such as tabs, most visited history on your homepage and bookmarking. It offers a very visual way of interacting with large data catalogues, such as movie databases or games catalogues, through film posters and game covers, something that I really love.

You can see Gary Flake demonstrating Pivot at the TED conference in Feburary here:

[more]

The interface is lovely and it does open up new ways of interacting with these large data sets, but there does seem like some work has been done to prepare the data for use in Pivot, to make it more visual, such as the very way the different endangered species are shown, or the Wikipedia categories are visualised. This isn’t a bad thing, but it does mean if the data isn’t already visual as with films or games then there is an interesting piece of design work to be done to take full advantage of this browser. I’d love IMDB to get an API as this would be an amazing way to navigate their site.

While I instantly enjoy the visual navigation style, this is still far from replacing my usual browser, because while it allows me to interrogate the data in new ways, I am not sure I would use this new way of examining data for the current sets available, after all I tend to dip into these data heavy sites only when I have a specific query on something, where searching is much more appropriate than browsing. While I do think that the way we search and find data will change dramatically I think this sort of interface could really come into it’s own on the sites I tend to browse, such as Amazon or Facebook and I think it could offer a really interesting dimension to Twitter.

Project Natal

Project Natal was announced by Microsoft at E3 as new interface peripheral for the Xbox 360, which removes the need for a joypad, instead body movement, facial recognition and voice are used for interaction.

We have seen similar innovations with the EyeToy for the PS2 and the notion detection in the Wii, but both have severe limitations. The EyeToy is a single lens camera, so it is easily affected by lighting and background, while the motion detection in the Wii is built into the joypad and has limited capabilities by default.

The demo shown at E3 is jaw dropping, showing seemingly natural interaction with a character called Milo, but anyone with any experience of Microsoft demos treats them with appropriate levels of scepticism. Since they haven’t beaten the Turing test there is certainly some smoke and mirrors going on, in the words of Milo’s creator Peter Molyneux “If we had, then applying it to a computer game would be the last of the solutions we’d use it for.”

[more]

Other demos include a painting application and a dodgeball style game. They are relatively low tech examples but the dodgeball game has been demonstrated with journalists and celebrities, and has appeared to withstand their scrutiny. Interestingly Endgadget’s demo turned the lights down to see if it can cope with different lighting, and it had no problems.

Although shy on the details Microsoft have said it contains an RBG camera, an infrared camera, a multi-array microphone and a depth map. New Scientist among others suggested it is using the infrared camera technology by 3DV, an Israeli company that Microsoft purchased. This heat detection allows it to maintain a level of functionality regardless of light sources or obstacles.

Project Natal claims to have facial recognition and voice recognition, but we’ve only really seen this in the Milo demonstration, which could have been entirely scripted. So the extent of this capability is still not fully understood, although the voice recognition is based on Windows 7 functionality.

While Natal does recognise actions sitting down, a notable thing from the demos is the space of the room they were demoing in. When you use your whole body as a controller you need enough space to do so. A bit like Wii Fit, removing the traditional controller can pose logistical problems for the smaller household; for me my living room becomes more like an obstacle course.

Paradoxically playing a game with your body rather than your thumbs can be both more intuitive and harder work. I am old enough that can still remember my initial awkwardness interacting with a computer mouse or a console joypad, even though it seems like second nature now. However this hurdle is removed by Natal, making gaming and potentially all human-computer interaction much more accessible and intuitively responsive to an even wider audience, although it lacks the feedback a physical peripheral affords.

If I sat playing Burnout as it has been demonstrated with Natal my arms would ache after a while. But one thing I do know from marathon Guitar Hero sessions is that I will play through most aches for the right game. Interestingly a great many of the games we play allow us to explore far beyond our own physical limitations, so I wonder where the disparity is between simulation and escapism and what we desire from our gaming experience.

I don’t think we will be getting rid of the joystick anytime soon, but this definitely opens the door to new kinds of games and game play that we haven’t seen before. Its greatest potential is in the creative and imaginative hands of games and software designers everywhere.

Seadragon on the iPhone

Microsoft has released it's first iPhone application and its a good one. Some of the more interesting things that are coming out of Microsoft labs are Photosynth and Seadragon, you can see an excellent demo of this at a TED talk from 2007:

In brief Seadragon allows you to zoom quickly and smoothly through a great deal of visual information, so much information infact that when I originally saw the demo I assumed it was a little bit of demo magic being worked on a very high end computer. The zoom interface of Seadragon would seem like a natural fit with the multi-touch interface of the iPhone, but I had always thought that the iPhone did not have the necessary computing power to run such an application, thankfully I was wrong.If you are lucky enough to have an iPhone or iPod check Seadragon out and let me know what you think.

The Magic of Interface Design

While reading Derren Brown’s book 'Tricks of the Mind' (thanks Simon) I began to see some overlap between performing close-up magic and designing user interfaces; it comes down to understanding and predicting your audience’s attention.

The approach however is from opposite sides of the problem; while the interface designer tries to focus the attention of the user to enable them to achieve a task, the magician relies on our inability to focus on everything to perform their trick. Especially when we are concentrating on something we can easily miss even the most obvious things, so while we are looking to see how the trick is done it distracts us from how it is actually done.

Try this awareness test:

It makes it clear that if there is a lot going on then not only is it harder to find the information you want, but you can miss important information you didn’t know you needed. In the context of interface design it can be frustrating for a user and ultimately may prevent them from being able to achieve their goal.

[more]

Learnt Blindness

We also learn not to see things through experience and expectation. An example of this I saw recently was putting a quick link to an F.A.Q. page in an area of a page normally reserved for advertising, as a result few users saw the the new link; we have become conditioned to expect advertising and therefore don't look at that area of the page. See Nielsen's research on this banner blindness.

As interface designers we need to consider that to break with expectation can mean we create something that can stand out, because it breaks from the it's surroundings conventions, however it might never get seen, such as when we put important navigation elements where we expect to see advertising. So we need to understand when it's appropriate to use the audience's own expectations to help them find the information they need.

Keep it simple

When dealing with an audience's attention it helps to remove distractions. Keeping an interface as simple as possible and removing unnecessary distractions, can help make it clearer and easier for the user to focus. But it's important to not throw out the baby with the bathwater, if you remove too much information then there aren't enough cues to know what to do next.

One time performance

This is important if it is an application that is used rarely or as a single experience. The reason you will rarely see a magician repeat the same trick is because it gives the audience the opportunity to see it in a different way, we know what is coming and we've already studied the trick once, if we see it again it makes it easier with each consecutive performance to see how the trick is achieved. This is the same for interface design if we know what to expect, through repeat use, it is easier for us to know what to do over time as we can handle more information and learn the interface. In which case we can adopt more complex interfaces, that offer more functionality, but take longer to learn.

TaDa…

In conclusion we need to understand that our awareness and attention are often compromised and easily distracted, and take this into consideration when designing user interfaces. A good experience is like a magic trick, we might not be aware of exactly how we got there, it seems easy, but we are delighted we did.