TouchIT

personal contact

Not only are we individually physical beings, living and acting in a physical world, but also our contacts with one another are played out in that same physical world whether in a formal handshake or in sexual intimacy.  Even our spoken words, gestures and facial expressions require physical proximity if not contact.

However, over the years, layers of technology have allowed us to establish contact while temporally or geographically remote, letters and telegraph, paintings and photographs.   Now-a-days the opportunities for remote contact often seem to swamp those for face-to-face encounters with Facebook, Second Life and even plain old-fashioned video-conferencing.  We will discuss virtual reality in a later chapter, but forms of video-conferencing belong here, as it is the image of the real human body that is projected across space, even though its transport is by pixel and wire.

Real face-to-face conversation is enabled not just by spoken words, but by continual gestures, and movements of the face and eyes.  While video preserves these better than text based media or avatar-based VR meetings, still the glass barrier of the screen not only separates psychologically, but may also lose some of the subtle cues that help us to communicate.

One problem is that when you look straight at the image of the person you are talking to, the camera is slightly to one side or above the screen, so that your conversant sees your eyes averted or looking down.  One early solution, the video tunnel, used two ways mirrors to enable the camera to be effectively placed in the middle of the screen [[SO91]].  However, it was an impractically large contraption, useful for experiments, but not real use.  Current laptops and screens often have a built in CCD camera close to the top of the screen, which reduces the angle compared with older ‘bolt-on’ arrangements, but still not perfectly.

The situation is worse still for multi-person conversations as it may not even be clear, who one is talking to.  The Hydra system (see Box [[**xref**]]) tackled this by having a number of tiny screens each with its own camera, so effectively each screen ‘stood’ for one of the participants on the desk [[SB92]].  The HP HALO System (see Box [[**xref**]]), is more sophisticated with identical rooms at different locations designed so that the video of the remote room seems to fit seamlessly into your own meeting room … bit of course at the cost of a dedicated and expensive installation.

intimacy

It is easy to think that everything virtual and Internet is new and never seen before.  It is always telling when technology finds its way into mainstream media; the film “You’ve Got Mail” [[Wa99]] did that for online romance, but virtual love affairs are now common, with Internet dating sites and meetings in chat rooms, and there have even been weddings in cyberspace dating back to MUDs in the 1990s [[Cu92, Ir96]].

However, romance by letter dates back hundreds of years and it was 1848 not 1998 that saw the first tele-wedding [[St98]].  The daughter of a wealthy US businessman fell I love with one of his employees.  To prevent the affair from developing the father posted the unwelcome suitor to then European office and he duly went.  However, the daughter was not to be foiled in love so easily and while her beaux was still in New York she recruited a priest, contacted him by telegraph, and while each were opposite ends of a telegraph line were married.  Her father contested the marriage, but the courts upheld the marriage.  Love conquers all.

Whether or not they were forged online, many couples and families are forced to have long distance relationships due to their jobs.  Email, instant messenger and web cams have now joined or supplanted the traditional love letter, but are not a substitute for physical closeness.  A number of more tangible ways of establishing physical contact have been exploited, some in artistic settings, some as research probes [[LM09]].

One early example consisted of two rollers linked to one another by the Internet [[**ref**]].  If the user at one end spun her roller then the roller at the other end would also spin.  If her remote partner spun his then she would see hers move.  And if they both touched the roller at the same time they could feel the pressure of the other either working with them or against them, like holding hands beneath the tablecloth.  Others have experimented with clothes using electrically reactive materials to create a hugging shirt, that lets you send ‘hugs’ to you partner using a mobile phone [[CC09]]. Of course, Kevin Warwick’s direct nerve implants attempt to establish a form of intimacy that, as far as possible, bypasses physical means entirely (see [[**xref**]]).

Perhaps most common have been various forms of connected jewellery; releasing fragrant smells, pleasant sounds or comforting glows at one end when the other end is touched or stroked.  This does not need to be for connecting lovers or partners, but others with close personal relationships.  One piece, ‘Journeys between ourselves’, focused on a mother of 75 and daughter of 45:

“Light sensors in each piece detect a wearer touching and holding the porcelain form. The partnering piece then softly trembles in response. We developed this haptic and tactile way of interacting with the neckpieces with the hope of facilitating a gentle, human centred mode of communication.” (from [[WJ07]])

‘Journeys between ourselves’ Jayne Wallace [[WJ07]]

As well as bringing remote people together, technology often gets in the way of direct human contact.  Central hearting replaces being together round a winter fire, and even the family TV watching has been replaced with per-person devices: mobile phones, iPods and YouTube.  In many households the Wii has taken the place of open fire or TV set as the heart of the family living room, except instead of furniture clustered round it is thrown out, spread to the periphery to allow the action of Wii in the heart [[No08, VG08]].  Whereas multi-person video games tend to favour remote interactions (even if different bedrooms in the same building), instead the physical action of the Wii begins to bring people together and has an appeal across ages.

mediation and sharing

Ethnographies of work settings repeatedly find that both physical things in the environment and also the arrangement of people in the environment are crucial to enabling coordination and collaboration.

One classic study was of one of the London Underground Control rooms [[HL92]].  There is a large display visible to everyone, the ‘fixed line diagram’, showing the locations of trains, track and signals.  Not only does this give explicit information, but the fact that it is commonly available is essential to the staff understanding one another’s behaviour.  As well as the fixed line diagram, which is updated by the actual movement of trains, the cellophane pages of a printed timetable are edited in felt pen by the Controller, a publically visible act and one emphasised by talking out loud.  In fact ‘over-hearings’ of one sort or another are critical for numerous activities in the control room, and are enabled by being distant enough to be able to get on with one’s own job, but close enough to be able to monitor other people’s actions and words.

Figure NNN.  London Underground Control Room from [[HL92]]

Studies of other ‘command and control’ situations such as the bridge of a ship, airport ground services, and air-traffic control reveal different details, but similar importance of mediating artefacts.  Air-traffic control has been particularly well studied in several countries due to the necessity to automate more due to increasing volume, and yet the difficulty of doing so in a way that does not compromise established practices and so threaten safety [[HO95]].

A critical part of the air traffic control system are flight strips, small slips of paper printed out when a plane enters the airspace.  These have information such as the flight number, and altitude of the plane.  The air-traffic controllers sit in pairs with their own screen, and the flight strips are placed in a rack between them. As well as updating the flight strips when, for example, the altitude changes, they also will slightly pull out strips that in some way are of interest.  This is partly to act as a reminder for the controller working with the plane, but also implicitly allows the other controller monitor the general situation; for example, lots of tweaked strips may mean problems.

Figure NNN.  Flight Strip (schematic)

It is not just in work life that we use the physical world to coordinate, in the home we have message boards, and notes stuck the refrigerator door; we notice that a coat is in the hall or keys on the table and so that a family member is home.  Many of these signs are implicit and we would be hard pressed to articulate all the ways in which we use them, but they are crucial in giving us a sense of ‘what is going on’.

With the coat in the hall it is the presence of the object that is the ‘message’, as opposed to things written explicitly on it as in the London Underground timetable.  We will discuss the different forms of these ‘accidental’ properties of objects in more detail in chapter [[**xref**]], but in this chapter we are interested in the way the presence, movement and exchange of objects facilitates social interaction.

When having a meeting one person may pass a report across the room, the report was there all along, but the act of passing it says “this is now for you to read”.  When sitting near someone, even changing the orientation of a paper may say, “you read this too, but I’m still holding on to it”.  When working remotely with digital media, we may be able to see someone on web cam, hear them and be able to send documents, but it is hard to replicate the subtle interactions around shared paper documents.

For people working together shared screens can be one means of establishing that rapport whilst using digital technology.  One example involved using a large horizontal screen or multiple smaller screens for planning Australian travel itineraries with a travel agent [[RR03,RS07]].  The shared point of reference and side-by-side orientation allowed many of the same subtle bodily interactions of paper-based meeting, but sill making use of the power of the digital planning software including interactive maps, hotel and tourist information.

Figure ZZZ.   The Oz travel planner [[RR03,RS07]]

Interactive tabletops such as the Microsoft surface allow more subtle interactions as the digital documents can be shuffled around not unlike real papers.  One system, UbiTable , uses a table top display that can link to users’ own laptops [[SE03]].  Although only a prototype the idea is that it would be used in public places such as an airport where people may have an ad hoc meeting and want to share some information, but keep others private.  With paper you might have papers in your briefcase, papers on your side of the table, or papers moved so the other person can see them.  UbiTable emulates this by connecting to your own laptop (private) and allowing documents to be moved to the shared tabletop display.  Once on the display they can be on your side of the table facing you (personal) or moved into the centre of the table and rotated around so the other person can easily see it (public).

Figure UUU.  An ad hoc meeting around the UbiTable [[SE03]]

socio-organisational Church-Turing hypothesis

Some organisations clearly have a physical purpose: a factory making cars, a bus carrying passengers, or a theatre presenting a play.  Others, whilst having physical premises, have less physical outputs: a bank, insurance company, or a travel agent (as opposed to an airline or hotel).  However, even those with more outputs also have ancillary functions more like the bank or insurance company: processing orders, paying staff, managing stock.  An organisation has many roles, but one of them is some form of information processing, either, like the travel agent, as its principle role, or, like the car factory, as a secondary one.

In computing the Church–Turing Thesis asserts that in the end all computers can do the same things given sufficient memory and time.  Strictly this is referring to what can be achieved or what is computable; in principle computers could be very different in the way in which they perform computation.  However, in practice most computers are very similar in terms of the major components and structure.  If you have a new computer you can ask questions like “where is its memory” and usually get a sensible answer.
In fact commonalities are also found between the human brain and computers, and this computer–brain analogy lies at the heart of much of cognitive science.  There are limitations to such a view, indeed our brains perform a substantial amount of associative thinking, linking ideas together by similarities, whereas digital computers tend to be largely sequential.  However, it has proved a very useful analogy for both disciplines.

Given this and the fact that organisations perform information processing functions, it seems reasonable to look for structural similarities between the organisation and a computer, a form of socio-organisational Church–Turing hypothesis [[DW08]].  In fact, distributed cognition (see section [[**xref**]]) uses precisely this sort of computational language when discussing individual and small group activity in their environments [[HK02]].

The most obvious two elements in a computer are the program and data.

Computer programs are always explicit (the code) and expressed formally.  Similarly the organisation will have some formally written processes and rules of operation.  The following of these rules is not however in a lock-step fashion across the organisation, but each department and each individual doing their job and together (hopefully) achieving the overall organisation’s objectives.  That is, in terms of the computational analogy, it is a distributed system. However, not all the rules are formally written down; the clerk just knows that when she gets a blue form ZK9b it needs to be signed and passed on to accounts.

The data is usually easier to locate: some stored physically in filing cabinets and ring binders, some digitally in databases.  This aspect of the organisation has been most consistently subjected to computerisation; although the process side is also automated in some workflow systems. Even here some of the data is in people’s heads, particularly more personal sales knowledge about customers and clients.

While the implicit rules data and processes may work well, for the organisation this can be a problem when and employee leaves or is sick.  The desire to record, preserve and pass on this knowledge has been a major impetus in several areas of management science such as organisational learning and customer relation management.

However, what about the more ephemeral aspects.  A computer has a  ‘program counter’ that says where in a program it is operating.  So for an organisation, how does it ‘know’ and remember what it is doing? Imagine it is night, everyone has gone home and the office lights are turned off.  The records of the organisation are there in filing cabinet and database.  The processes in rule books and people’s job knowledge.  But in the morning how does the organisation ‘pick up’ where it left off the night before? Somewhere there need to be placeholders that say what is to happen next.

Just as with the processes and the data, some of this may be explicitly written down, perhaps ticks on a timetable of activities or to do lists left on people’s desks.  Some also will be in people’s heads, they remember one day where they left off the day before.  However, typically much is stored in the physical locations of documents, records and forms.  You know you were part way through filling out form ZK9b yesterday because this morning it is sitting there on your desk half complete, and the organisation knows that that form is at a particular point in its processing because it is on your desk not somewhere else.

culture and community of practice

When we think of culture, particularly past culture, it is likely to be in terms of physical artefacts and buildings: the Mona Lisa, the Elgin Marbles, pyramids and Acropolis, MacDonald’s burgers and Marvel comics.  For oral cultures studied by archaeologists there are indeed only the material traces; the society that created the cave paintings of Lascaux may have had a sophisticated oral culture with folk tales and sing for which the familiar running horses and aurochs were just a back-drop.  But if so the words and music died with them, all we have is the paint on the walls.

Cave painting Lascaux, France

Our view of our technological past is likewise informed and often relies solely on preserved or discarded artefacts, from Neolithic axe heads to Victorian shovels.  Even in relatively recent times when the records of the great and the letters of the middle classes preserved their legacy, the lives of mundane craftsfolk are poorly documented; we know them through what they made.
In a literate age, print and music stave have preserved words and music although even now the knowledge of things can easily be lost (see box).

The things we have around us from the TV set to the electric kettle, shape our understanding of who we are, and how we see the world.  However, the Soviet philosopher Ilyenkov went further, he saw ‘ideals’ (ideas or concepts) as being both embodied in artefacts and also giving those artefacts meaning [[Il77]].  So a chair is a chair and not simply pieces of wood because the ideal of the chair has been made material through the labour of a carpenter and has become a chair.  However, equally it is our idea of the chair that enables us to make sense of those pieces of wood arranged in the way and see that they are for sitting on.

political

The physical world is central to politics.  The land itself is at the core of the notion of the nation state: whether protected by walls and border controls, or simply measured out on a map.  However, when we consider national identity it is often the symbolic nature of physical things that become important: the Houses of Parliament, Capital Hill or Kremlin, waving flags and national costume.

Digital technology has challenged the physical form of the nation state making border permeable.  In the demise of the Soviet Union, the Allied bombing of Bagdad, contested elections in Iran, and numerous G8 summits, the openness of the Internet challenges the power of arms and police.

The subversive power of the digital is perhaps most radical in its destabilising of the control of information.  As Orwell’s 1984 highlights, what we have most to fear is not so much the physical power of states, but the more insidious psychological repression [[**ref**]].  Again and again totalitarian states have reduced physical distinctiveness to de-individuate and de-personalize and have used the control of the press to control people’s mind.  In 1984 even the dictionary was re-written so that only the ‘right’ information could exist.  Blogs make the faceless human and tweets undermine propaganda.

However, the digital world also gives the very tools of state that Orwell imagined, in the UK there is one CCTV camera for every 15 people and whilst steaming open letters was laborious and hard to hide, the digital scanning of a nations email is silent and swift [[Wo06]].

In other ways globalisation is reducing the integrity of the nation state, which was historically rooted in space and geography.  Multinationals, campaign groups, NGOs and social networks all claim loyalty; if Facebook were a nation it would be the 5th most populous in the world.

Groups have always claimed allegiances before and beyond the nation state.  Some groups are regional, for example Italy is a young nation and feel regional identity, even down to the town (a sort of local nationalism called ‘campanile’ checking this), before feeling Italian.  However, just like the modern trans-national groups, others are non-geographic based around religion, trade or now-a-days sport.

In nature, computation and in human society there are two ways in which something can obtain integrity and identity.  One is through boundaries: cell membranes, city walls, or barbed wire fences, and the other through shared attributes: ant pheromones, or tagged photos in Flickr.  Where groups have no geographical boundary to define them, they often resort to physical symbols to identify them: coats of arms on the mediaeval battlefield, hoodies in the shopping mall, or brightly stripped scarves in the football stadium.   In the digital world there are no boundaries and no physical, but blogs and myspace pages are skinned and decorated, and it is possible to buy badges and T-shirts based on your favourite computer games.

Figure RRR.   Physical symbols in a digital age  (http://retro-stuff.co.uk/)
football scarves image (rangers v Celtic Steve?

References

[[SO91]]  R. B. Smith, T. O’Shea, C. O’Malley, E. Scanlon and J. Taylor. Preliminary experiments with a distributed, multimedia, problem solving environment. In J. M. Bowers and S. D. Benford, editors, Studies in Computer Supported Cooperative Work, pages 31–48. North-Holland, Amsterdam, 1991.

[[SB92]]  Sellen, A., Buxton, W. & Arnott, J. (1992). Using Spatial Cues to Improve Videoconferencing. Proceedings of CHI ’92, ACM Conference on Human Factors in Software, 651-652.

[[Cu92]] Curtis, P. Mudding: Social Phenomena in Text-Based Virtual Realities, Intertek Vol 3.3 1992.  Text at http://scara.com/~ole/literatur/mudding.html and http://w2.eff.org/Net_culture/MOO_MUD_IRC/curtis_mudding.article

[[Ir96]]  Aisling Irwin. When the anoraks wear mink.  Time Higher Education Supplement, 3 May 1996. http://www.timeshighereducation.co.uk/story.asp?storyCode=93510&sectioncode=26

[[Wa99]]  Warner Bros. Pictures. You’ve Got Mail. 1999. http://youvegotmail.warnerbros.com/

[[St98]]  Tom Standage. The Victorian Internet: The Remarkable Story of the Telegraph and the Nineteenth Century’s On-Line Pioneers.  Walker & Company (October 1998) ISBN-10: 0802713424  (see also review by Eric Goldman, “The Victorian Internet”, November 16, 2007  http://blog.ericgoldman.org/archives/2007/11/victorian_inter.htm

[[LM09]]  Lottridge, D., Masson, N., and Mackay, W. 2009. Sharing empty moments: design for remote couples. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 – 09, 2009). CHI ’09. ACM, New York, NY, 2329-2338. DOI= http://doi.acm.org/10.1145/1518701.1519058

[[CC09]] CuteCircuit LLC.  The Hug Shirt.  (accessed June 2009).  http://www.cutecircuit.com/projects/wearables/thehugshirt/

[[WJ07]]  J. Wallace, D. Jackson, C. Ladha, P. Olivier, A Monk, M. Blythe, and P. Wright. Digital Jewellery and Family Relationships. Workshop on the Family and Communication Technologies, Newcastle, UK, 2007.

[[No08]]  N. Nova, LIFT-Labs.  tangible@home. MobiKUI2008, First International Workshop on Mobile and Kinetic User Interfaces, Fribourg, Switzerland, 13-14 Oct. 2008.

[[VG08]]    A. Voida, S. Greenberg, “Wii All Play: The Console Game as a Computational Meeting Place”, Research Report 2008-912-25, Department of Computer Science, University of Calgary, Calgary, Alberta, Canada, September 2008.

[[HL92]]  Christian Heath and Paul Luff.  Collaboration and Control: Crisis Management and  Multimedia Technology in London Underground Line Control Rooms Journal of Computer Supported Cooperative Work,  Vol. 1, No. 1. pp. 24-48. 1992.
[[HO95]]  Hughes, J., J. O’Brien, M. Rouncefield, I. Sommerville and T. Rodden  (1995).  Presenting ethnography in the requirements process.  In Proc. IEEE Conf. on Requirements Engineering, RE’95. IEEE Press, pp. 27–34.

[[RR03]]   Rodden, T., Rogers, Y., Halloran, J., and Taylor, I, (2003) Designing novel interactional workspaces to support face-to-face consultations. Proc CHI’03. ACM Press. 57-64

[[RS07]]  Y. Rogers, H. Sharp, J. Preece.  Interaction Design: Case Studies: 11.1: Supporting Collaboration for choosing Holidays. (dated 2007) http://www.id-book.com/casestudy_11-1_support.htm

[[SE03]] Shen, C.; Everitt, K.M.; Ryall, K., UbiTable: Impromptu Face-to-Face Collaboration on Horizontal Interactive Surfaces, ACM International Conference on Ubiquitous Computing (UbiComp), October 2003, pp. 281 – 288

[[DW08]]  Alan Dix, Julie Wilkinson, Devina Ramduny (1998). Redefining Organisational Memory – artefacts, and the distribution and coordination of work.  Workshop on Understanding work and designing artefacts, York, 21st September 1998. http://www.hcibook.com/alan/papers/artefacts98/

[[Il77]]   Ilyenkov, E. Problems of Dialectical Materialism, (Translated by A. Bluden). Progress Publishers, 1977. http://www.marxists.org/archive/ilyenkov/works/ideal/ideal.htm

[[Wo06]]  D. Wood (ed.). A Report on the Surveillance Society. Report For the Information Commissioner by the Surveillance Studies Network . Information Commissioner’s Office, UK.  September 2006 http://www.ico.gov.uk/upload/documents/library/data_protection/practical_application/surveillance_society_full_report_2006.pdf