TouchIT

When something stops working and is safe to do so, you might give it to a child as a plaything. Or maybe you are waiting for a bus and have a restless, you might give your phone to the baby to play with (but after turning it off so that she does not accidentally call the police!).

With an iPhone the baby could perhaps use it as a mirror, feel the weight of it, look at its shininess. With a more traditional phone, the baby might press buttons, perhaps open and close the lid.

Figure PPP. (i) a nice mirror (ii) buttons to push and (iii) slide the phone in and out

When we think of a device such as a phone, we quite rightly treat it as a whole “I press this button and it dials a number”. However, as we started to see at the end of the last section and the playing baby demonstrates, the physical device has interaction potential even when unplugged, disconnected from its power and digital functionality.

Think of the phone without its battery, or tearing a central heating control off the wall and snipping its wires. What can you do with them? What do they suggest to you?

As the baby would discover, the iPhone on the left in Figure PPP has very little interaction potential without its power: there is one button at the bottom of the screen, and few small buttons on its edge, all artfully placed to not obscure the clean lines of the phone. In contrast, the phone on the right has a variety of actions that can be performed, pressing buttons, sliding the keyboard in and out.

For any device we can examine this interaction potential when unplugged and then separately see how these physical actions map onto its digital functionality.

Exposed state

One of the simplest examples of a physical device is a simple on/off light switch. In this case the switch has exactly two states (up and down) and pressing the switch changes the state (Figure LLL). Note that even this simple device has interaction potential, you can do things with it.

Figure LLL. light switch: two states — visible even when the light bulb is broken

Actually even this is not that simple as the kind of press you give the switch depends on whether it is up and you want to press it down or down and you want to press it up. For most switches you will not even be aware of this difference because it is obvious which way to press the switch … it is obvious because the current state of the switch is immediately visible.

Note that the switch has a perceivable up/down state whether or not it is actually connected to a light and whether or not the light works — it has exposed state.

The phone in Figure PPP also has some exposed sate in that you can see whether it is open or closed, but the buttons are not the kind that stay down. The iPhone has no exposed state at all. Here are some more exposed state devices (Figure EEE).

The sockets are similar to the light switch except here the red colour on the top of the switch is also designed to give some indication of the mapping; that is feedforward. The washing machine control dial is more complex, but again it is immediately obvious by looking at the dial that it has many potential states. Like the power switch it also tries to provide feedforward through words and symbols around the dial. We will return to the washing machine dial later as it has a particularly interesting story to tell.

The central heating control is like the more mobile phone as it has a flap that moves up and down. Like the light switch or this means there are tow very visually and tangibly obvious states “open and closed”. However, this is a very particular form of exposed state as its effect is to hide or reveal other controls. In this case the purpose is to hide complexity, but it may also be used to protect against unintended actions — when the phone is closed it is impossible to accidentally dial a number. In the case of the phone there is of course yet another purpose, which is to change the form-factor, when closed the phone is smaller to fit in your pocket or handbag.

Hidden state

In contrast to these exposed state devices, consider this volume control on a CD player. It has clear action potential, perceptual affordance: you can see that it sticks out, is round, it invites you to pull, push and, in particular given its roundness, twist it. However, remember the power is unplugged and so there is no sound (or imagine twisting it during a moment of silence between movements). There is no indication after you have twisted it of how far. The washing machine and cooker knobs were styled and decorated so there was an obvious “I am pointing this way” direction, but here nothing. In fact, inside things have changed, but on the outside, there is nothing detectable; it has hidden state.

Figure CD. Volume knob on CD player — no visible state

Another common example of hidden state are bounce-back buttons, such as often found for the on/off switches of computers. Consider the TV and dishwasher button in Fig. DWTV. Superficially they look similar, however when you interact with them, their behaviours differ markedly. With the dishwasher button you press it and it stays in (in fact, the ‘on’ position when the power is on — see the little red light, the power was actually on when the photo was taken!); that is it has exposed state. In contrast you press the TV button in and as soon as you let go the button bounces right back out again. Of course the TV turns on or off as you do this, but the button on its own tells you nothing; that is hidden state.

Figure DWTV (i) dishwasher with exposed state out=off, in=on (ii) TV hidden state bush-back button exposed state switches: visible difference up/down in/out

If this seems a minor thing, maybe you have had the experience when the TV screen is blank, but you don’t know whether this is that because it is off, because it is in standby, because you have turned it on and it is warming up or because the DVD player connected to it is off? In fact, if you have learnt to decide hem it is often possible to see because small red LEDs are added — in this case you can see the LED next to the button labelled ‘STAND BY’. However, in reality, do you really look at all those little red lights or do you simply press a few buttons at random on the different boxes until something happens?

Maybe you have even lost data from your computer because you accidentally turned it off when it was in fact just sleeping? On many computers, both desktop and laptops, there is a single on/off button. To turn it on you press it, to turn it off you press it, but it simply sits there looking the same. You open the laptop or look at the and monitor (which itself maybe because the computer is asleep or because the monitor is). Thinking it is off you press the power button often to then, too late, hear the little whirr as the disk started to spin as it woke form sleep, only to hear the dull thud as it turns off and starts to reboot. What was onscreen before it went to sleep? Did you save the draft of that chapter on annoying hidden state buttons?

Figure HHH. Computer on/off button: hidden state with power light

As you contemplate several hours lost work, you can take comfort in the fact that the designer has often foreseen the potential problem. In the photo above, you can see that this computer button, like the TV button earlier, has a small light so that you can see that the power is on, in this case a small green (unlabelled) LED. If you had been observant, if you had realised this is an indicator meaning “turned on” rather one meaning “connected to the power”, if you hadn’t got confused in the moment, then you could have worked out it was on and not lost all that work — small comfort indeed.

Now there are good reasons for using a bounce-back switch, which we will discuss in detail later, one of which is when the computer can also turn itself off in software. However, these bounce-back buttons are often found on computers when this is not the case (indeed the one photographed in Fig HHH does not have a software ‘off’) and an old fashioned up/down power switch might be more appropriate, or one where, like the dishwasher button, it stays depressed when in the ‘on’ position. Even where the software can switch the power off, why not simply have an ‘on’ button and then an additional ‘emergency off’ button for the cases when the software is not shutting down as it should? This could be small and recessed so it is not accidentally pressed, rather like a wristwatch button for setting the time.

Sometimes the reason for not doing this is simply lack of insight, and sometimes plain economics — the cents or pence it cost to add an extra button are worth more to the manufacturer than your lost work! However, often it is aesthetics: your lost work is weighed against the flawless smooth casing with its single iconic button. And if you think the designer made a poor choice, what do you think of when you buy a new computer? It is a brave designer who is willing to focus on the long-term benefit of users that improves their life, rather than the immediate appearance that makes them buy the product. Are you brave enough? Or maybe it is possible top achieve aesthetics and safety, certainly the additional small ‘emergency off’ button could be located slightly out of sight (although not so hidden the user can’t find it!), or maybe made and essential part of the aesthetic of the device.

Tangible transitions and tension states

When you twist the DVD knob shown in Figure [[DVD]] it is heavy to turn, giving it a feel of quality however apart from that no sense of how far you have turned it. However, not all knobs and dials are like this.

Figure PPP shows three experimental prototypes that were produced for a photo viewer. All have an area where a small screen would go and all have a rotary control. The one on the left has a very obvious retro dial, the middle a more discrete dial, and the one on the right an iPod-like touch surface. In all the prototypes rotating the dial enables the user to scroll between different menu options, although never more than seven at any level.

While they all use rotary controls, they feel very different in use. On the right the touch surface offers no resistance at all, your finger goes round, but without the display you cannot tell there is anything happening. In fact, it is perhaps only because one is sued to devices like this that one would even try to stroke it — the cultural affordance of the iPod generation! In contrast the more clunky looking prototype on the left has a far richer repertoire of tangible feedback. It already has exposed state as you can tell what direction it is pointing, but in addition has end stops so that you can feel when it has got to one extreme or other of the menu. Also, the mechanics of the mechanism mean that there is slight resistance as you move between its seven positions; that is it has tangible transitions between states.

Figure PPP. Three prototype phone viewers

Tangible transitions are particularly important when considering accessibility for the hearing impaired or for occasions when you cannot look at the screen, such as when driving. The left hand device has both end-stops and tangible transitions; this means that once someone has learnt some of the menu layouts, the device can be used without looking at the screen at all. Even when you can see, the tangible transitions give additional feedback and the resistance between the positions makes it difficult to accidentally select the wrong option.

The device in the middle has a form of tangible transition, as there is a very slight sensation as one moves between positions, but it has no end stops and there is no resistance before it moves to a new position. The lack of resistance makes errors more likely and the lack of end stops means it is harder to orient oneself except by looking at the screen, however at least it is possible to tell how many steps one has taken.

It is not only knobs that can have tangible transitions. The light and power switches discussed earlier not only have a visible state, but there is definite resistance as you push the switch down, it gives a little, and then the sudden movement as it flicks down. If you release the pressure of your finger before it flicks down, it simply bounces back to where it started. In a sense the device has at least four states; as well as the obvious up/down as there are also part up and part down states as one pushes the switch, although only the up/down states are stable when you release your finger pressure.

Figure XXX. Light switch with part-way states

Bounce-back buttons, such as the computer power button in Figure HHH, can similarly be seen as actually having two states out and in. Only the out state is stable, but while you press with your finger it remains in a pressed-in state. This is a tension state, one where you have to maintain a continuous effort in order maintain it. In the case of the computer power, the tension is never maintained, you just press and release. However, tension states are often used as part of interaction, for example, dragging with a mouse.

Keeping you hand, or other muscles in tension can cause fatigue if maintained for a long period, and also affects accuracy and timing. Indeed, Fitts’ Law measurements show measurable differences in performance between ordinary mouse movement and dragging [[**ref**]]. However, the advantage of a tension state is that you are very aware that you are in the middle of doing something. When typing it is possible to break part-way through a sentence and leave it incomplete, however it is impossible to go away part-way through dragging the mouse, you have to release the mouse button and end the drag. This can be particularly important in safety critical situations, such as the use of the ‘dead mans handle’ in trains.

Natural inverse

It is summer holidays, and you are driving down a small country lane in Cornwall where the sides of the lanes are usually high earth banks and the lanes themselves winding, narrow and with no room for a car to pass. The car is packed full with suitcases and tents, spades and swimming costumes, so you cannot even see out the back window. Suddenly, round the bend ahead another car appears coming towards you. You both stop, one of you must go back. There is a relatively straight part of the lane behind you with nowhere to pass, but you do recall you passed a gateway just before the last bend, so you shift the gear into reverse and begin to edge backwards, with only your wing mirrors to see where you are going.

At first you drive very slowly, everything is back-to-front and unless you think very hard you turn the wheel in the wrong direction. However, after a bit you find yourself confidently driving backwards at a fair speed down the straight lane behind. Every so often you suddenly ‘lose’ it, end up getting too close to one of the banks and have to stop, and, as if from the beginning, work out which way to turn the wheel, but each time you quickly get back into the flow.

Even if you are a very experienced driver and find reversing is no longer difficult, maybe you have tried to reverse a trailer or caravan and had a similar experience.

It is reasonable that this is difficult if you are not used to reversing a car long distances, but what is remarkable are the periods in between when it becomes easy. It is not that you have learnt the right thing to do, as you find that when you get out of the flow, you have no better idea which way to turn the wheel than when you started.

The reason for these periods when it becomes easy is that the steering wheel exploits very basic human responses — the natural inverse. When you draw a line on paper and decide it is in the wrong place, you need to find the eraser and rub it out. It is not hard, something you will learn to do without thinking, but is something you have had to learn. If however you are trying to put the pencil inside a desk-tidy and move your hand a little to far to the right, you automatically move it slightly to the left. In the world there are natural opposites: up/down closer/further, left/right, and our body mirrors these with muscles and limbs hard-wired to exploit these.

Rodolfo Llinás’ work showed that some of this is very low-level indeed, with sets of mutually inhibitory neuron’s that allow pairs of opposing muscles to be connected [[Li02, p.45]]. Although higher-level brain functions determine which pairs operate, some of the actual control even happens in the spinal column as is evident in the headless chicken that still runs around the farmyard. These paired muscle groups allow rhythmic movement such as walking, and also the isometric balancing of one muscle group against another that is needed to maintain a static position, such as holding a mug in mid air. maybe drawing on neurons from [[Li02]]

When you first start to drive in reverse, you have to think to yourself “if I want the car to move to the left which direction do I need to turn the wheel”. However, when you have started to move wheel and discover it is going in the wrong direction, or you are about to overshoot and go to far, you do not have to thing again, but instinctively move it in the opposite direction. The natural inverse takes over and you do the opposite of what you were doing before.

Unplugged devices often have buttons, knobs and other controls that have natural inverse actions: twist let/twist right, push/pull. The minidisk controller in Figure CCC was intended to be clipped on to clothing while the user was walking or running. Given it will be used eyes-free it is particularly important that the physical format helps make it easy to use. The device has two different kinds of control and both of them exhibit natural inverse.

On the end of the device is a knob. Twisting the knob in one direction moves on to the next track, twisting it in the other direction moves it back a track — natural inverse. The knob can also be pulled out and this changes its function: twisting it one way increases the volume, twisting it in the opposite direction turns the volume down. Note also that this is a tension state, it is obvious whether you are changing track or changing volume, not just from the immediate aural feedback, but also because the knob wants to spring back into place, to adjust the volume requires continuous tension.

Along the side of the device are a number of spring-back sliders; they can be pushed forward or backward along the device. Each slider controls a different function, but all of them use the same principle. There is an ordered list of options for each setting; pushing the slider moves between the relevant option settings one way through the list, pulling it back moves it the other way. Note that the natural inverse reduces the impact of mistakes. If you choose the wrong option and change it, you instinctively move the slider in the opposite direction and restore the setting.

Figure CCC. Minidisk controller

Using the natural inverse can obviate the need for explicit ‘undo’ operations, and can make a control usable even when you don’t know what it does. The phone in Figure PPP belonged to one of the authors for some years. On the top left-hand side of the phone is a small slider. This slider did different things in different modes: when in a call it adjusted volume, when in the address book it scrolled through the names. The author never knew exactly what it did, yet used it extensively. This was because it always respected the natural inverse property and so he could use it without fear; if it didn’t do what he expected he just did the opposite movement and carried on.

References

[[Li02]] Rodolfo Llinás (2002). I of the Vortex: From Neurons to Self. MIT Press.

1. N.B. not to be attempted on the open road if you value your life! [↩]