User Interfaces in Games

Games UI Series

For some time I have written about both my professional and social interests on this blog; covering user experience and gaming, but I want to combine them and look at user interface design in games. I think this is an oft-neglected part of games, especially with the usual budget and time constraints, however as with any software design the usability of the user interface can have a profound effect on the user’s experience.

World of Warcraft screen with massive campaign detail

An advanced user experience on World of Warcraft

Usability in games is not restricted to on screen interactions, there is a such diversity of ways to interact with your gaming platform of choice; be it joypad, keyboard, touch screen, or no controller at all. This makes the platform and method of interaction a key part of the user experience in games, as such I will explore the strengths and weaknesses of these human-computer interfaces.

Some games designers and developers think that creating games is completely different to creating other software, because they are creating entertainment rather than tools. However recently as we have seen an increasing overlap between games and applications e.g. Epic Win we can see that these lines are far more blurred than previously considered. Software development has only recently realised the commercial value of user experience, but games developers often consider themselves the audience as well as the creators, failing to realise that their familiarity with their game hampers their ability to see their product impartially; perhaps more frustrated by the focus groups that require them to “dumb down” games than they are in the issues that may cause that confusion in the first place. While games do need to offer challenges in order to evoke a sense of achievement, these challenges should be designed and deliberate and not a hurdle of a poorly designed interface.

I was delighted to see that Edge has added to its staff Graham McAllister; the CEO of Vertical Slice, the UK’s first usability testing company to focus solely on games. This recognition of the need for usability in an industry leading publication can only help raise the profile of the value of understanding your users.

I’m hoping to write a series of game reviews, which look specifically at the UI and give a heuristic review on their strengths and weaknesses as well as offering possible alternative solutions where appropriate. Read More

Android and HTC: The mobile power couple

I recently acquired a HTC Desire running Android 2.1 and it has significantly changed the way I use my mobile phone. The primary reason for this can easily be attributed to Android. I knew when selecting a new phone that the apps would make or break it.

So why not go with iPhone? Because HTC make the best hardware on the market.

HTC Desire

[more]

Hardware vs Software

I have been a fan of HTC handsets for several years now, as they regularly push the hardware limits of mobile devices, squeezing in just that bit more functionality than their competitors; however more often than not the software was a letdown, not living up to the potential of the hardware, even in things as crucial using it as a phone.

Apple blew the market away; the hardware is not as groundbreaking, but it made sure that the OS fully supported the hardware functionality, concentrating on the usability of the product in a market that was saturated with terrible UI design.

Windows Mobile

A few months later Android launched.

Apps

The reason my mobile usage has changed is largely because of Marketplace. My old HTC Diamond, has amazing hardware, but runs Windows Mobile and the apps are sold in a more traditional software business model (as you might expect on a Microsoft platform) with a price tag to match. Apple created the AppStore, as a single point for all apps, accepting micropayments through iTunes.

Android has easy access via their Marketplace and also utilises a lot of the other successful Google applications like Gmail and Google Calendar, meaning for me it is taking the tools I’m already using and seamlessly putting them in my pocket. I could check my Gmail on my old Diamond, but it was pull not push and didn’t support HTML; now it is easier to use my phone than it is my PC.

Apps are purchased through Google Checkout with a 24 hour refund policy, so rather than enforcing heavy restrictions on the apps available as with iTunes, users can try and review apps without cost for 24 hours.

Widgets

Android has a desktop like space which you can customise with widgets, which means you don’t need to launch applications to get to useful information, such as your appointments, photos or if your train is on time.

HTC and Android

HTC have a history of taking off the rough edges of the OS interface with HTC Sense. This made my old HTC Diamond very usable and it hid a myriad of Microsoft’s sins behind smooth animation and gradient interface menus. With Android this work isn’t needed, allowing them to concentrate of creating some great looking apps and widgets; that means your phone is ready to run straight from the box, no need to go hunting through the Marketplace for Facebook or Twitter apps, just sync and go.

Development

Fragmentation has always made mobile development difficult and while neither platform is particularly easy to develop for, Apple have recently taken the step of excluding apps that have been compiled from Flash, which could reduce the number of developers with the necessary skills to create apps. Android is Open Source and Java based as opposed to Apple’s Objective C, and there is an increasing market of tools to help compile for your desired OS. Apple no longer has the largest market share, although the expected release in June of a new handset will most likely boost iPhone sales again.

The cost of submitting an app is considerably cheaper for Marketplace than it is any other platform, and the volume of apps is increasing as the Android adoption grows.

Handsets and Networks

Apple has at last allowed networks other than O2 sell the iPhone, but the hardware doesn’t change until Apple release new version. Android is available on many different handsets, so you can pick one that best suits your specific needs. HTC are constantly developing new handsets so you can always get the most cutting edge technology on the market.

Pivot – interesting data navigation from Microsoft Labs.

Pivot is a tool for browsing large collections of data, such as Wikipedia, and really shows the value of a good API and semantic data.

It contains a filtering system on the left to help you dig into and out of the data, combined with a smooth zoom interface as we’ve seen implemented before in Seadragon and Photosynth and some very familiar browser elements such as tabs, most visited history on your homepage and bookmarking. It offers a very visual way of interacting with large data catalogues, such as movie databases or games catalogues, through film posters and game covers, something that I really love.

You can see Gary Flake demonstrating Pivot at the TED conference in Feburary here:

[more]

The interface is lovely and it does open up new ways of interacting with these large data sets, but there does seem like some work has been done to prepare the data for use in Pivot, to make it more visual, such as the very way the different endangered species are shown, or the Wikipedia categories are visualised. This isn’t a bad thing, but it does mean if the data isn’t already visual as with films or games then there is an interesting piece of design work to be done to take full advantage of this browser. I’d love IMDB to get an API as this would be an amazing way to navigate their site.

While I instantly enjoy the visual navigation style, this is still far from replacing my usual browser, because while it allows me to interrogate the data in new ways, I am not sure I would use this new way of examining data for the current sets available, after all I tend to dip into these data heavy sites only when I have a specific query on something, where searching is much more appropriate than browsing. While I do think that the way we search and find data will change dramatically I think this sort of interface could really come into it’s own on the sites I tend to browse, such as Amazon or Facebook and I think it could offer a really interesting dimension to Twitter.

Project Natal

Project Natal was announced by Microsoft at E3 as new interface peripheral for the Xbox 360, which removes the need for a joypad, instead body movement, facial recognition and voice are used for interaction.

We have seen similar innovations with the EyeToy for the PS2 and the notion detection in the Wii, but both have severe limitations. The EyeToy is a single lens camera, so it is easily affected by lighting and background, while the motion detection in the Wii is built into the joypad and has limited capabilities by default.

The demo shown at E3 is jaw dropping, showing seemingly natural interaction with a character called Milo, but anyone with any experience of Microsoft demos treats them with appropriate levels of scepticism. Since they haven’t beaten the Turing test there is certainly some smoke and mirrors going on, in the words of Milo’s creator Peter Molyneux “If we had, then applying it to a computer game would be the last of the solutions we’d use it for.”

[more]

Other demos include a painting application and a dodgeball style game. They are relatively low tech examples but the dodgeball game has been demonstrated with journalists and celebrities, and has appeared to withstand their scrutiny. Interestingly Endgadget’s demo turned the lights down to see if it can cope with different lighting, and it had no problems.

Although shy on the details Microsoft have said it contains an RBG camera, an infrared camera, a multi-array microphone and a depth map. New Scientist among others suggested it is using the infrared camera technology by 3DV, an Israeli company that Microsoft purchased. This heat detection allows it to maintain a level of functionality regardless of light sources or obstacles.

Project Natal claims to have facial recognition and voice recognition, but we’ve only really seen this in the Milo demonstration, which could have been entirely scripted. So the extent of this capability is still not fully understood, although the voice recognition is based on Windows 7 functionality.

While Natal does recognise actions sitting down, a notable thing from the demos is the space of the room they were demoing in. When you use your whole body as a controller you need enough space to do so. A bit like Wii Fit, removing the traditional controller can pose logistical problems for the smaller household; for me my living room becomes more like an obstacle course.

Paradoxically playing a game with your body rather than your thumbs can be both more intuitive and harder work. I am old enough that can still remember my initial awkwardness interacting with a computer mouse or a console joypad, even though it seems like second nature now. However this hurdle is removed by Natal, making gaming and potentially all human-computer interaction much more accessible and intuitively responsive to an even wider audience, although it lacks the feedback a physical peripheral affords.

If I sat playing Burnout as it has been demonstrated with Natal my arms would ache after a while. But one thing I do know from marathon Guitar Hero sessions is that I will play through most aches for the right game. Interestingly a great many of the games we play allow us to explore far beyond our own physical limitations, so I wonder where the disparity is between simulation and escapism and what we desire from our gaming experience.

I don’t think we will be getting rid of the joystick anytime soon, but this definitely opens the door to new kinds of games and game play that we haven’t seen before. Its greatest potential is in the creative and imaginative hands of games and software designers everywhere.