This is a book about understanding the physicality of people and of technology and how studying these can help us to better design physical technology for people.
Why Study Physicality
Historically there has often been a dualistic view of mind and body with cognition and thought being seen as quite separate from physical muscle and bone. Computation has a similar abstract feel, a world of algorithms and pure reason.
However, both are embodied in flesh and silicon, and if we wish to communicate with any computational device, we do so through the physical world; whether tapping keys, pushing a mouse or stroking an iPhone screen. Not all these interactions involve fingers and touch: speech involves tongue and mouth creating pressure waves through the air that vibrates the microphone in the computer, gesture recognition involves moving arms, photons and optics, and even direct brain interfaces use electrodes and wires. There is no connection in pure abstraction but always in the physical world.
As more devices around us have digital aspects, it becomes essential that we understand the way physical and digital interactions meet. Traditionally the physical form of a product was the domain of the industrial designer whereas the digital interaction belonged to usability or human-computer interaction specialist. However, now, in products from mobile phones to washing machines, the physical design and the digital design must come together to create a single experience for the user.
How many computers in your home?
How many computers in your home? Probably several depending on how many people in your house, maybe if you are very geeky, you might have separate development or gaming machines, maybe even old machines for nostalgia’s sake. However, do you have a HiFi, microwave, TV, washing machine? Each of these typically contains one or more computers; so how many computers in your home? Think about the answer, maybe if you are geeky you may need a quick run around and count. Later in the chapter, we’ll return to this and see how you compare with the authors
Components of the physical world
The world can be seen in many ways, by biologists, physicists, geographers and more. Focusing on those aspects that influence the design of hybrid digital/physical devices, we are organising this book around four main themes of the physical world
- I the body and physiology — Our bodies are physical as are our brains; however numinous our thoughts they have their life in the material substance of neurons. Our bodies too can be part of digital interactions, whether gaming in the living room with Wii or using an advanced gesture recognition interface.
- II material artefacts and design — The world is full of ‘things’ both natural things such as stones and constructed things such as scissors, books and mobile phones. Our understanding of the former is germane to the design of the latter.
- III space and spatial arrangement — When we interact, we do it in physical space. This space may be where we perform physical movements – living rooms have been reorganised to accommodate the Wii [[No08],VG08]]; but it also has social dimensions. For example, at Lancaster we have the Hermes systems; small displays outside our office doors where visitors can leave notes [[CF03]]. We have never had problems with abusive messages, which is probably because anyone leaving a note is aware that they could be observed in the public corridor – the spatial location changes use [[DC04]].
- IV digital artefacts and virtual physicality — Sometimes we emulate aspects of the physical world in the digital: virtual reality creating whole parallel worlds; the desktop metaphor; dragging images in an iPhone; or even the idea of ‘visiting’ a web site. Do we understand enough about the physical world to be able to capture the right aspects? And computation itself is embodied in silicon and magnetic surfaces and bound by that materiality (see also the PalCom project “making computing palpable”[[PC08]]). There can only be a finite amount of computation in a finite space, and information flows take time, hence the star like pattern of supercomputer circuits. Even the Turing machine can be thought of as a touring machine as any physical attempt to pull an infinitely long tape would break the tape and the machine instead needs to traverse a fixed tape – the tape embodying finite information in finite space and the machine with finite computation.
These themes are not independent: as we have seen, as humans we interact with computation through the physical world and most commonly through devices, and always set within space whether in offices, homes, streets, countryside or open sea.
Of course the world is more than these things, there are animals and plants, fire and storm, but the above four seem to be those most intimately connected with digital devices … although there have been proposals to help people interact better with their pets! [[TL06, Mc09]]
Kinds of Things: from stones to silicon
While this book is about the confluence of digital technology and physical design, in fact digital technology is the latest in a long process whereby humankind has shaped its world. With the exception of the last, each of the major themes above encompasses both natural phenomena and artificial ones. Figure 1.1 lists some of the kinds of things we find in the world divided left to right depending on whether they are natural or artificial and also grouped into their themes, where they fit into one.
the natural and artificial world
In fact, the distinction into natural and artificial is itself slightly problematic, after all we are part of the natural world, so in a sense a computer is as natural as cow dung, both products of animals in the world. If we exclude bodily functions, then certainly a bird’s nest should be included as an artificial construction. Similarly, we have put language on the artificial side of the divide, although equally bird song could sit there. Both lines and language have a tendency to impose discrete distinctions upon continua, a point we will discuss later in the book.
the natural order
The Ancient Greeks had four basic elements earth, water, air and fire, although Aristotle added a fifth aether, for the material of the heavens. Similar systems occur across the world. While earth represents the solid things of the world, water, air and fire are increasingly numinous, hard to hold or contain, or even see.
Of the kinds of things listed for the natural world in Figure 1.1, most are things of earth, the solid things we can see, touch and hold: the landscape, stones, plants, animals and other people. While all solid, they differ dramatically in the way we understand them and the way in which we can interact with them.
The most ‘earthy’ of natural things is the ground itself beneath our feet and the landscape that stretches out around us. This forms the matrix within which we live and act hence the importance of ’space’ in the organisation of this book.
The landscape near and far is not bare but full of inanimate and living things. The inanimate whether never-living such as stones, or once-living such as sticks or shells, are our start point when we look at material artefacts, and in some ways the simplest of things: in our control, not changing unless they break. When we want a child to learn we often give them simple blocks of wood (or plastic) to play with, where they can experience and create order or be the agent of their own chaos.
Plants are not so different from stones except it is these living and once living things that tend to be more pliable, changeable in form not just location: even wood becomes more solid as it ages. Animals, however, change the rules; just like other people they move and act on their own volition. In contrast, just like stones, the ground or dead bones, a tree will stay where it is. Interaction with stones and plants depends only on your own actions, but to interact with an animal (whether chasing it to eat, or running away to avoid being eaten) you need to consider what it will do.
Any attempt to distinguish people from animals based on essential attributes, whether consciousness, self-consciousness, intention, intelligence or moral sense, is bound to lead to debate if not argument. However, problematic such distinctions are, we humans are special in this book; indeed it is we who create the digital technologies and it is we who will use them.
Returning to water, air and fire; the first is more well-behaved, tending to leak to the earth if given a chance, but otherwise controllable, holdable, movable. No wonder the liquid metal in Terminator 2 is so frightening and the water horses in Lord of the Rings so awe inspiring. However, we don’t need to look to cinema for this as the crashing sea or flowing stream have also always held that sense of ‘otherness’ and mystery. But it is air that is most often associated with life itself, and even spirit; in Genesis God breathes into man and the blowing wind apparently gives vitality to trees and even dead leaves.
Fire on the other hand is the destroyer, but also perhaps the most mysterious of all. From a scientific point of view it is just hot glowing and burning gas, but as ordinary people, it seems to defy rules, apparently with edges yet constantly forming and reforming. As water seeks to escape downwards, it constantly flows upwards to the sky, but never gets there disappearing with at best a dark dirty smoke left from its pure glowing heart.
This said, it is perhaps fire that gives us the best metaphor for many of the more abstract notions of language, society and computation. It is essentially an emergent phenomenon. Whilst earth, water and air are in the end composed of molecules and atoms, a fire is defined not by a particular set of molecules, they are constantly changing coming from the fuel and going into smoke. The identifiable flame is more about the self-sustaining form and structure in which these passing molecules find themselves. The heat of the fire vapourises the fuel, the hot gases burn, the combustion creates heat. There is a boundary below where the air and fuel flow in and a boundary above where there is no more combustable gas, oxygen or the heat is too diffuse. But it is not a boundary you can touch, not even indirectly with a stick or metal rod. Like air and water, fire sometimes seems to have a life of its own, from the terror of a bush fire to a ‘living’ flame gas fires in the hearth.
This nature of fire as constantly changing and yet also fixed and bounded was germane to Heraclitus’ philosophy, which in turn influenced the views of later Greek philosophers including Aristotle. Heraclutus saw that this change was the essence of even things appearing unchanging, or, as Karl Popper phrased this in more modern philosophical language, Heraclutus sees everything as process [[Po63]].
Our bodies are of course similar, we eat food, and shed skin; we have all heard that each breath we take contains an air molecule once breathed by Julius Caesar. Like fire, it is not the particular molecules that make us who we are, but the arrangement. Flocks of birds, multi-national companies and files on a magnetic disk all share this. However, now we have moved on to the artificial world, the world forged by human hands.
the artificial — works of our hands
The roots of the word ‘artificial’ are from the Latin ‘ars’ meaning art and ‘facere’, to make; the artificial is the made. The Chinese elements differentiate earth, wood and metal: although small amounts of precious metals occur naturally, the last of these is largely a product of human labour. We not only shape the earth, but add materials to it that do not even exist or at best, rarely exist in natural form. Arguably fire itself should have been in the category of the artificial; whilst natural fires exist on earth as the result of lightening or volcanoes, still the majority of terrestrial fire is man-made. It is fire that allows the smelting of metal, fire that drove the industrial revolution and the carbon dioxide from that fire that threatens to destroy us now. With stone tools and skeletons, it is fire circles that characterise the oldest remains of hominids.
Whilst not the oldest traces of humankind, it is the remains of buildings that are often the most impressive signs of the past: the Acropolis in Athens, the Coliseum in Rome, the Great Wall of China, the hilltop city of Machu Picchu in Peru. Indeed of the ancient Seven Wonders of the World, five are buildings and the other two giant statues. Buildings change our relationship to the environment, we can shelter from the weather, be protected from animals and other humans. Large scale building also requires sophisticated human organisation and planning; many of the four ton stones used in the construction of Stonehenge were transported hundreds of miles from Pembrokshire in South Wales over a period of many decades.
The Seven Wonders of the World
- Great Pyramid of Giza
- Hanging Gardens of Babylon
- Temple of Artemis at Ephesus
- Statue of Zeus at Olympia
- Mausoleum of Halicarnassus
- Colossus of Rhodes
- Ishtar Gate of Babylon (earliest lists)
- Lighthouse of Alexandria (later lists)
This list was compiled by the Greeks as a sort of Lonely Planet Guide, so it is not surprising that all are in or near to Greece. More recent lists include wonders from other parts of the world. Only one of the wonders exists today, the Great Pyramid of Giza, which was also the oldest at around 2500 BC
make photo grid??
|Pyramids at Giza||The Ishtar Gate|
However, it is not just buildings that influence our relationship with the natural landscape. Often larger still are the works of civil engineering such as the Roman roads that cut across Europe, or more recently the Suez Canal, or the trans-Siberian railway. Indeed the Suez Canal shows how powerful a political issue these re-mouldings of the geography can be. Of course, the Roman roads were there precisely to enable fast movement of troops to conquer and to control; and it was the railroad as much as the US Cavalry that enabled the expansion of the western frontier and eradication or displacement of the native American tribes.
In many areas of the world, agriculture provided the surpluses that enabled the growth of large social units. However, the public works needed to maximise production also led to political and physical change. In Egypt the whole civilisation is believed to have arisen in order to harness the annual floods of the Nile through massive works of embankments and channels. Even what we now consider to be natural environments are often the result of large scale agricultural interventions; the Lake District in Northern England is shaped partly by work of nature ice and rain, but would be wooded if it were not for the sheep.
In later chapters we will see that humans are not the only tool-makers, but we are certainly the most prolific and sophisticated. The earliest tools were axes and spears to hunt, fish and prepare food and bone needles to fashion clothes. In fact textile production has always been one of the most technically sophisticated activities; from early handlooms to the Spinning Jenny, which was the catalyst for the industrial revolution.
As technology advanced, both mechanical and electrical things have, in their time, been seen as almost magical. This is perhaps in part because they seem to defy the laws that hold for simple physical objects: hidden mechanisms and harnessed power. In each age humankind seems to have painted itself in the image of these technologies with artificial humans made in clockwork, then later steam and electronics. From Pygmalion to Golem to Frankenstein, literature and folktale has stories of made-humans becoming live, and in science we use images of the day to make sense of or bodies and minds. Descartes imagined tiny movement of the nerves carrying information from fingers to brain; and today we cast the brain in information processing language.
It is precisely this computational and digital technology that is the focus of this book; to some extent numinous, like air and fire, hard to put your finger on and touch. However, we shall see that the notions of information are there from the earliest stage of modern humanity and intimately tied to language itself. Information and language are carried by particular words on the page, sounds in the air, or electrons in a wire. Just as the constituent molecules of fire change from moment to moment, the words are passed on from person to person. Each tongue of flame is unique, but the fire itself has a dynamic but persistent form. The words also are in a sense different in each telling, but the power of language and the power of information is that they also have a meaning beyond the particular wax tablet, parchment or silicon chip.
Revisited: How many computers in your home?
How many did you count? Probably several depending on how many people in your house, maybe if you are very geeky, you might have separate development or gaming machines, maybe even old machines for nostalgia’s sake.
Well our counts were:
|Jo||21||(and she lives on a boat!)|
Yikes, are we total geeks? Well, if you haven’t guessed already, think again; do you have a HiFi, microwave, TV, washing machine? Each of these typically contains one or more computers; so how many computers in your home?
Try looking around your body too. How many computers do you wear, have in your pockets, or carry around in your wallet, or handbag? Phones, digital watches, chipped credit cards, cameras, even car keys all have computers or some form of computer technology in them..
This book comes as the confluence of two areas of research and practice: Human–Computer Interaction and Product Design. Product design is about generating ideas and solutions for physical products whether in the home (kettles and corkscrews), in the workplace (adjustable chairs, paperclips and filing cabinets) and in the world (cat’s eyes on roads, or waste paper bins on the high street). Human–Computer Interaction (HCI) is concerned with any place where people interact with technology. This may be a one-to-one interaction between a person and single computer (as in the person typing on a laptop to write this), or may be a situation where the computer is a mediator in a human–human communication (Skype calls and instant messaging).
For many years those working in HCI had to largely treat the hardware of computing (keyboard, screen and mouse) as a ‘given’ and the focus was almost exclusively on the design of software and understanding on the cognitive, social and organisational impact. In contrast, product design was confined to objects with at most relatively simple mechanical or electrical activity. However, as devices such as washing machines and mobile phones became more complex in terms of their interactivity product designers started to attend HCI conferences, and as computers became embedded in everyday devices those in HCI started to look to product designers for insight on physical design.
Making things usable — Human Computer Interaction
The roots of HCI can be traced back at least 50 years. In 1959 Brian Shackel published probably the first HCI paper on the ergonomics of displays [[Sh59]] and the early 1960s saw the development of landmark systems including Ian Sutherland’s Sketchpad [[Su63]] and the work Douglas Englebart’s Bootstrap Institute [[EE68]]. The Bootstrap Institute was home to the development of concepts that took many years to become mainstream, including video conferencing and hypertext, and, perhaps most iconic, the first computer mouse. However, it was in Sketchpad that saw the first use of the computer screen to create an interface that was in some way like the physical world, with a light pen used to edit diagrams on-screen.
|SketchPad in use (from [[Ka09]])||SketchPad light pen (from [[Su63]])|
The late 1970s and early 1980s saw the emergence of a recognisable discipline of Human–Computer Interaction, Shackel again being critical in establishing the international community. This expansion was largely driven by the emergence of the personal computer and in particular the Apple Macintosh, still seen as a design icon as well as an enabling technology. The visionary element was still there, especially in the work of Xerox PARC Laboratory that pioneered many of the interaction techniques that found their way into the ‘Mac’.
The first Macintosh [[He09]]
What’s in a name.
In the early days HCI was known by different names, including the “Man–Machine Interface”. The latter term had various problems, “Man” is gendered, “Interface” fails to give regard to broader interaction and context, and “Machine”, well, just sounds dated. However, the focus on “Computer” was perhaps unfortunate as “Machine” took a broader view of technology. In these days when computers are in everything it is perhaps a moot point, and the name has not stopped the field embracing issues of technology in a broader sense whenever people interact with it. However, for this book, one wishes the field could be called Human–Technology Interaction as the computer, while present in almost all modern technology, is not the focus of human interaction with it.
For many years the field focused almost exclusively on interaction through the graphical user interface, itself a physical metaphor at use in the flat digital world of a pixellated screen (discussed more in chapter [[**xref**]]). However, there has always been a strand that looked at physical interactions, often focused on the design of improved or novel input devices, including squashable balls, foot-based interaction and numerous devices for navigating in 3D environments. Studies of technology in context have always emphasised the importance of the physical layout of work environments; for example, the different ways practice nurses, general practitioners and hospital consultants oriented their screens to include or exclude their patients [[**ref**]]. However, it is in recent years, with the development of mobile technology, ubiquitous computing and tangible interfaces (see later) that physicality has become a core issue in HCI.
Of Designers, Information Appliances and Physicality
What do we mean by design? It is a word that means many things to many people. Cambridge Advanced Learners Dictionary defines it as “to make or draw plans for something, for example clothes or buildings” [[**ref**]] a definition that leaves plenty of room for its application in a range of areas by a very broad range of professionals and amateurs from chefs to chemist and from electronic engineers to ergonomists. Not only that, its meaning has also changed over the years: “mass production has evolved and has been perfected, and in the course of this evolution the designer has been variously an artist, an architect, a social reformer, a mystic, an engineer, a management consultant, a public relations mans and, perhaps, now a computer engineer” [[BC07, p. 55]]. For the purposes of this book we shall be concentrating on one area of design; the design of physical objects, in particular but not exclusively, those designed for production (i.e. industrial design, more on this below).
Arguably design is as old as people: whether fashioning flint arrowheads or building brooches. Generally speaking, what separates modern design from design before, say, the Victorian era, is that the earlier design tended to be based more on individual output rather than mass or even batch production. There are exceptions. The Romans, for example, produced prelaid mosaics that could be bought ‘off the shelf’ for installation in the home. Examples of ‘mass production’ can even be arguably found in Neolithic times: Skara Brae is a Neolithic village on the principle island of Orkney, Scotland. It consists of ten dwellings dating to around 3100 BC. They are very well preserved being largely intact with the exception of their roofs. The buildings themselves are of a standardised layout with furniture, cooking area, beds and dressers all in the same relative positions in each dwelling. The stone furniture itself is also of a standard design.
Figure x: Stone dresser, Skara Brae
Mousa Broch, Shetland
For most of recorded history ‘design’ tended to be within single disciplines such as crafts (pottery, jewellery), engineering (bridges, canals) or architecture. Again there are exceptions such as the detailed drawings of da Vinci and the Renaissance confluence of arts, science and (albeit anachronistic to say) engineering. Throughout the later middle ages, in the increasingly mercantile society, the craft guilds grew although largely to protect the secrets of their crafts not critique or develop them; and more aesthetic traditions in high-end furniture, fashion and architecture have flourished from the Renaissance on.
Design and manufacture began to separate in the mid 18th Century when mass production began to be used. Josiah Wedgwood was among the first to bring in sculptors to ‘design’ the form of the ceramics the Wedgwood factory manufactured [[BC07, p. 17]]. The ability to reproduce forms accurately meant that the quality of the end product was determined by the quality of the design at the beginning. It was in the late 19th century that more reflective and modern design traditions had their roots. Just as today, design philosophies varied greatly. On the one hand John Ruskin’s works lead to the nature-inspired and handcraft focused Arts and Crafts movement. Whereas Ruskin and the Arts and Crafts movement were partly acting in reaction to the mechanism of the industrial revolution, others embraced the utilitarian lines of new materials and methods giving rise to Modernism. The architect Louis Sullivan was one of these, both developing the steel-structure high-rise and also coining the design maxim “form ever follows function”. This period also saw the rise of the first mass-produced electrical appliances, with irons, toasters, cookers and electric fans appearing all before 1910 [[Ga07]].
According to Bayley and Conran, the term ‘Industrial design’ first appeared in 1919 in America, introduced as a method by which products could compete essentially through ‘styling’ when their prices were ‘stabilised’ during the Great Depression. The profession quickly expanded its role to embody product function as well (e.g. Dreyfuss’s home phone in 1937). The International Council of Societies of Industrial Design (ICSID) no longer attempt to define industrial design. However, in 1963 they defined it as “a creative activity whose aims are to determine the formal qualities of objects produced by industry. These formal qualities are not only the external features but are principally those structural and functional relationships which convert a system to a coherent unity both from the point of view of the producer and the user. Industrial design extends to embrace all the aspects of human environment, which are conditioned by industrial production.” (ICSID 2009).
Another key influence in the development of industrial design as we know it today was the Bauhaus. Founded in 1919 by an architect, Walter Gropius, the Bauhaus was a ground breaking art school that sought to bring arts, craft and design together in a single modernist movement.
Adopting Sullivan’s ‘form follows function’ approach, important Bauhaus designers such as Mies van der Rohe designed products and buildings that exploited new materials and processes that relied on simplicity and mechanical integrity for their structural and aesthetic success.
|Home Phone by Henry Dreyfuss (1937) image source: http://www.telephonymuseum.com/new_page_1.htm accessed 12:51, 7th October 2009||Brno Chair by Mies Van der Rohe (1929-1930) image source: http://www.designicons.co.uk/img/products/Ludwig-Mies-van-der-Rohe-BRNO-Chair-1261-1.jpg accessed 13:00 on 7th Oct 2009|
Industrial design’s importance grew through the 20th century alongside industrial production including the foundation of the American Union of Decorative Artists and Craftsmen in 1927, the British Council of Industrial Design (now the Design Council) in 1944 and ICSID in 1957. Together these various movements and societies shaped a profession whose role was to cohere a design solution that answered the needs of all stakeholders to create the finished item with which the user interacts. Within that process they were required to understand the user’s limitations, abilities, desires and frustrations. A good industrial designer should, it follows, design products with an appropriate cognizance of their context of use.
In 1980 Heskett pointed out that one of the outlooks that separate the industrial designer from the artist is how the form is realised. An industrial designer will not make the objects they design. However much they may use physicality in the development process it is merely a means to an end, yet the act of making is fundamental to an artist. Key to developing these understandings is the methods used. Typical tools include sketching, maquettes, soft models and rigs. All of these are used for iterative development. Sketching is the fastest method of recording and developing ideas, but industrial design is a three dimensional activity and physicality is at the core of good industrial design practice. Even the most experienced designers need to trial their ideas physically, in three dimensions. Lawson notes: “The drawing offers a reasonably accurate and reliable model of appearance but not necessarily of performance. Even the appearance of designs can be misleadingly presented by design drawings. The drawings which a designer chooses to make whilst designing tend to be highly codified and rarely connect with our direct experience of the final design” (Lawson, 1997). Which is another way of saying that frequently only designers can understand the drawings they create and that these drawings can fool everyone including the designer that a design solution has been reached.need full Lawson ref
|Sketching and brainstorming||From sketch to lo-fi prototype|
The physical world is a complex place and three dimensions and material physicality can rarely be successfully modelled in the mind or the sketch pad. So for the development of form, proportion, aesthetics and structure designers supplement two dimensional sketch development with maquettes (scale models, generally of large products) or soft models, typically using card or modelling foam. Soft models often create the first opportunity for a user to interact with the product’s physicality, and this is a key component of the design process where a concept will frequently stand or fall. Ulrich and Eppinger note: “industrial designers build models of the most promising concepts. Soft models are typically made in full scale using foam” (Ulrich and Eppinger, 1995). The other three dimensional development technique, the rig (sometimes also known as the prototype), is used to develop the mechanical aspects of a design. Rigs don’t always prototype the whole of the design and they sometimes won’t look like the design concept. Their function is to test concepts’ mechanical properties, mechanisms or user interaction: “prototypes are built to test and validate the design” (Kai et al 2008). For example a rig of a chair concept would be strong enough for users to sit on and would be adjustable so that the designer could swiftly act on observations regarding seat height, angle, comfort etc. but it wouldn’t necessarily look like the finished article.
need full references Ulrich, Kai, Suri
“A few years ago it was simple. Designers designed things: objects like lamps, chairs, computer mice…”
(Suri, 2004, p. 13)
Full size prototypes
The mixture of physicality, cogitative and drawing based techniques described above have served designers well for decades (or millennia, depending on your definition of design). However the ascendance of the transistor has created challenges that the design community have yet to satisfactorily integrate into those tried and tested methodologies. As far back as the early 1990s designers started to realise that the products we now know as information appliances were going to pose challenges because much of the human interaction with them occurred on the physical-digital border. Steve once had a conversation with an interactive appliance design expert who told him that, in his experience, industrial designers tend to address user interfaces, particularly the graphical user interfaces of information appliances, with a 2-D mentality. Considering industrial design is all about three-dimensional output that’s a damning criticism, but should it surprise us? This enforced disjoint in the design process for products with computers in them is key to the theme of the book and so it will be a strand we pick up later in more detail.
Different ways to touch
There are a number of related areas, which study the meeting of physical and digital worlds. Each overlaps either HCI or product design to some extent and we will encounter them all at different times during the rest of this book.
- ubiquitous computing (ubicomp) — In 1991, Mark Weiser painted a vision of computation permeating the world, woven seamlessly into life [[We91]]. His thinking was based on early prototypes at XEROX research centres in Palo Alto and Cambridge, mostly involving displays of various sizes from 1 inch ‘tabs’ to yard scale wall displays and explicit interactions. More recent work often also uses forms of implicit sensing (e.g. cameras) and less ‘in your face’ ambient displays. Despite Weiser’s vision, in fact, technology often seems far from seamless, but certainly both our body-load of devices and the number of digital devices in our homes suggests that they are ubiquitous if not invisible
- tangible user interfaces (TUI) — In contrast, tangible user interfaces attempt to make computation both visible and touchable, where physical tokens represent digital things, embodying computation [[IU97]]. For example, a town planning application can use small models for planned buildings placed on a table on which a map is projected. As the planners move the models their location is tracked, a simulation works out projected traffic volumes and projects the resulting flows onto the map.
- mobile and personal devices — Of physical devices in day-to-day life, it is probably the mobile phone that has transformed lives most over recent years. Mobile applications can be divided into two main categories: (i) those where location doesn’t matter and (ii) those where it does [[BC07]]. This sounds like a tautology, but is a little more subtle! The first are those such as phoning, or accessing the internet, where the crucial thing is to transcend location. These are less interesting for the topic of this book. However, the other category is very relevant, when the physical location of the phone is central to the application as in maps, or ‘find a friend’ applications.
- virtual reality — Whereas ubicomp, TUI and mobile applications all involve actions in the physical world, virtual reality attempts to emulate the physical world within a digital environment. This may be a representation of a real or planned physical setting, as in tourist applications or an architect’s fly-through; may create a realistic, but artificial world as in the Sims or Second Life, or may use a physical world metaphor to represent information artefacts such as the computer ‘desktop’ or more sophisticated systems such as the Tower project that created an urban city-scape where each building and area mapped to files and folders [[PP04]].
Tower creates a 3D city of information ( http://tower.gmd.de/)
- alternative reality and mixed reality — Alternative and mixed reality systems blend the physical and digital worlds, overlaying digital imagery or other outputs (sound, tactile) on top of the real world, using spectacles, projectors or mobile devices. For example, a tourist guide device can show reconstructions of buildings overlaying archaeological ruins. Core to all is that there is some form of registration between physical and digital worlds, and so digital content can only be accessed at physical locations (similar to the distinction in mobile applications).
- physiological computing — While most of the above use the physicality of the device and the world as their connection point to the digital, there are also those for whom their very body has become the computational device. In assistive technology, small muscle movements and even nerve impulses are used to drive and sense prosthetic limbs [[Ke09]]. Physiological signals such as heart rate can be used in therapeutic settings or even to modify or control gameplay [[GD05]]. Notably Keven Warwick has famously (or infamously) voluntarily embedded chips in his body in the belief that our future as humans will be cyborg [[Wa03]], and artists such as Stelarc have explored this in performance [[ST09]]. We will return to these issues in chapter [[*xref*]].
Kevin Warwick wired for interaction [[Wa03]]
- human–robot interaction — Robots are slowly finding their way out of science fiction, and even out of the factory and into human lives: they include autonomous vacuum cleaners, the Sony AIBO and Philips iCat. Sometimes these are designed to do useful jobs, for example carrying cleaning equipment for an elderly person, allowing them to continue to remain autonomous in their homes [[Al09]]. Others, such as the AIBO, are designed for their emotional aspects; it is just a small cute puppy. The Phillips iCat is in some ways similar, but does not move around like the AIBO, instead its complexity is focused on its facial expressions and related features.
- telepresence robots – In addition to being used in one’s own home, the iCat’s can be networked, so that stroking the iCat in your home can make your friend’s iCat react; social robotics. More direct telepresence robots allow a remote operator in a limited way to ‘feel’ socially present somewhere else. These use technology similar to the remotely-controlled drones that are so much part of modern warfare, but instead applied to enhance social relations. Telepresence robots usually include a video camera and screen so that the remote operator can both see and be seen. However, unlike a video-conferencing the robot can be driven by the remote operator, so that they have a degree of movement.
In densely populated Japanese cities, it is hard to keep a pet. So interactive dolls and pet-like robots have been designed as companions for elderly people who look after them and often will not be parted from them, taking them out shopping or to see friends [[BB05]].
Roomba autonomous vacumm cleaner
The HeadThere Giraffe: Mobile Video Conferencing Robot
After these visions of our cybernetic future, the confluence of product design and human-computer interaction may seem prosaic, merely concerned with those mundane devices that fill our homes and pockets: MP3 players, mobile phones, washing machines, and TomToms. However, these are the actual things that fill our homes and pockets, not merely gadgets in research labs. As ubiquitous, tangible and mobile technologies become used technologies, they become part of product design, and then the challenge becomes how to turn them into things that can become part of our lives.
Learning about Physicality
So, where do we turn to learn about the nature of physicality
Most obvious are the sciences of physics and applied mathematics, which study the properties of the physical world, human physiology for the body and perhaps cartography or geography for space, and we will draw on knowledge and literature from all of them. These fields can tell us how a device will behave when acted on with a certain force, or whether a certain movement is possible or likely to cause injury. However, this is only part of the story; it may be possible for me to open a box, but will I do it, do I understand that I can do it and how I can do it. To be able to analyse and to design even non-digital products we need to understand how people understand. Understanding people is more the domain of the human sciences; psychology, sociology and anthropology, although they have very different methods. Psychology, on the whole, tends to use laboratory or other forms of controlled experiments, whereas sociology tends to study people in real life by various means from simple questionnaires to direct observation. All of these have proven useful in HCI in general and in those studying phenomena associated with physical interactions.
In Chapter [[*xref*]] we will discuss experiments targeted at understanding the ‘natural inverse’, the way some bodily movements are the opposite of others (push–pull, left–right). Because these involve low-level and often involuntary movements it is close to the stuff of traditional psychology and amenable to laboratory experiments. Elsewhere we make extensive use of knowledge gained from ethnographic studies, both our own and those of others. Ethnography, a technique drawn from anthropology, is about studying people in as near a natural environment as possible (insofar as simply having some form of observation always changes things). While the laboratory experiment is about creating a closed, controlled environment, ethnography embraces an open and often chaotic world, which very often reveals the subtle and complex ways in which people get mundane things done.
Archaeologists often have little to work with except the material remains of cultures and yet manage to work out much about the ways people live and work from that. In a similar way objects themselves tell us a lot about the way they have been used and the way they have been designed. Studying artefacts can be used as an ethnographic technique, and in Chapter [[**xref**]] we will see the way documents can be analysed to understand office practice. However, because artefacts have been designed they in a sense embody the knowledge of the designer, and in Chapter [[**xref**]], we will see how studying everyday consumer electronics can tells us about heuristics for physical interaction.
In both cases, it is often the case that implicit practices and knowledge can be found through analysing the artefacts; that is things that a person either didn’t know at all, or is so ‘obvious’ that they would never think to mention it if asked. For example, if you ask a person about their job, they will often tell you the ‘rule book’ version, but if you pick up a piece of paper on their desk and ask them how it got there and what would happen if you moved it, then a far richer story often emerges.
The very everydayness of physical interaction can make it hard to study. When you put a cup down, you assume it won’t walk away (and would be surprised if it did!), but normally you would not tell someone that this and other even more basic properties of the physical world are part of your implicit understanding of it. The problem here is that these things are not just obvious to the person being studied, but also to the analyst who is trying to study them. In order to see through this ‘obviousness’ of everyday things you need to make them in some way strange or odd.
In diverse areas including neurology and nutrition, it is when things go wrong that scientists begin to understand how they normally work. For example, the discovery of the role of vitamins often arose from recognising the deficiency diseases (see box [[**xref**]]). Where the effects aren’t harmful to health, it is possible to deliberately ‘break things’ in order to understand how they work. For example, the ethnographer Garfinkel encouraged his students to perform ‘breaching experiments’ such as standing really close to people while having an ordinary conversation; by breaking the social norms exposing how important they are [[Ga67]].
In diverse areas including neurology and nutrition, it is when things go wrong that scientists begin to understand how they normally work. For example, the discovery of the role of vitamins often arose from recognising the deficiency diseases (see box). Where the effects aren’t harmful to health, it is possible to deliberately ‘break things’ in order to understand how they work. For example, the ethnographer Garfinkel encouraged his students to perform ‘breaching experiments’ such as standing really close to people while having an ordinary conversation and by breaking the social norms exposing how important they are [[Ga67]].
Similarly, we have found various forms of ‘breaking’ have helped us in the necessary estrangement of the ordinary. The experiments on the natural inverse, which we will descirbe Chapter [[**xref**]], are of this kind, creating situations where pushing back and forwards on a joystick does not have opposite effects on the interface. This can also be used in more qualitative studies; in one study of group design we forced groups to use particular materials even if they weren’t the most appropriate for them (Chapter [[**xref**]]).
Fairytale, myth and science fiction are also rich resources as they reveal which elements of reality are necessary to human understanding of the world [[Dx00,Dx09]]. If we examine theses stories we find that some parts of reality are bent or broken. For example, fairytales often have magic doors that take you into different worlds, like science fiction has portals or teleportation, all breaking the normal contiguity of space. However, if everything is broken we have chaos, and if we look more carefully we see that the story makers retain certain properties, suggesting these are more crucial to our internal models of the world. As designers this tells us what we should retain and what we can afford to lose in our own emulations of, or interventions with physicality.
Of the Prevention of the Scurvy
A ship’s surgeon James Lind gave an early account of a quite rigorous experiment to determine the best treatment for scurvy (vitamin C deficiency) [[Li53]]. He took twelve patients with severe scurvy “putrid gums, the spots and lassitude, with weakness of their knees”, divided them into pairs and gave each patient one of six different treatments. It was the patients that had access to oranges and lemons, who recovered most quickly and completely
“The consequence was that the most sudden and visible good effects were perceived from the use of the oranges and lemons; one of those who had taken them being at the end of six days fit four duty. The spots were not indeed at that time quite off his body, nor his gums sound; but without any other medicine than a gargarism or elixir of vitriol he became quite healthy before we came into Plymouth, which was on the 16th June. The other was the best recovered of any in his condition, and being now deemed pretty well was appointed nurse to the rest of the sick …”
It was only in the 20th century that Vitamin C was isolated and fabricated, but the understanding of Vitamin C and its importance for health were the results of early studies like this of disease and deficiency
[[Al09]] James Alexander. Robots that Help the Elderly. eldergadget.com. May 14, 2009. http://eldergadget.com/robots/robots-that-help-the-elderly/
[[BB05]] Japan’s hi-tech carers. BBC News. 12 April, 2005. http://news.bbc.co.uk/1/hi/programmes/this_world/golden_years/4436633.stm
[[EE68]] Douglas C. Engelbart and William K. English. A Research Center for Augmenting Human Intellect. AFIPS Conference Proceedings of the December 1968 Fall Joint Computer Conference, San Francisco, CA, Vol.33, pp. 395-410 (AUGMENT,3954,).
[[Ga07]] Carroll M. Gantz. Design Chronicles: Significant Mass-Produced Designs of the 20th Century. Schiffer Publishing, Ltd., 2007. ISBN: 0764322230 Excerpts in “100 Years of Design”. Industrial Designers Society of America. (accessed July 2009). http://www.idsa.org/absolutenm/templates/default.aspx?a=51&z=28
[[Ga67]] H. Garfinkel (1967), “Studies in Ethnomethodology”, Polity Press, Cambridge
[[He09]] Andy Hertzfeld. More Like A Porsche. Folklore.org. March 1981 (accessed July 2009) http://folklore.org/StoryView.py?project=Macintosh&story=More_Like_A_Porsche.txt
[[Ka09]] Ivan Sutherland’ Sketchpad, with comments by Alan Kay., YouTube (accessed July 2009). http://www.youtube.com/watch?v=495nCzxM9PI
[[Li53]] James Lind. A Treatise of the Scurvy in Three Parts. Containing an inquiry into the Nature, Causes and Cure of that Disease, together with a Critical and Chronological View of what has been published on the subject. A. Millar, London, 1753. (transcribed Lars Bruzelius) http://www.bruzelius.info/Nautica/Medicine/Lind(1753).html
[[Ke09]] Duncan Kennedy. New robotic hand ‘can feel’. BBC News. Sunday, 18 October 2009. available at: http://news.bbc.co.uk/1/hi/sci/tech/8313037.stm
[[Po63]] Karl R. Popper. Kirk On Heraclitus, And On Fire As The Cause Of Balance. Mind 1963 LXXII: 386-392; doi:10.1093/mind/LXXII.287.386
[[Sh59]] Shackel, B., 1959. Ergonomics for a Computer. Design 120, 36-39
[[BC07]] Stephen Bayley and Terence Conran (2007). Design: Intelligence Made Visible. Conran Octopus Ltd.
[[Su63]] Sutherland, I. E. (30 January 1963). “Sketchpad: A Man-Machine Graphical Communication System” (PDF). Technical Report No. 296, Lincoln Laboratory, Massachusetts Institute of Technology. available at http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf
[[Wa03]] Kevin Warwick (2003). A study in cyborgs. Ingenia – Issue 16, Jul 2003. The Royal Academy of Engineering.