Position Paper for VRML '95

Abstract. A central challenge to VR technology is the need to support social interaction in multi-user virtual environments. To meet this challenge, we will need to define semantic frameworks in terms of which people can convey to each other what they are up to. This paper considers some of the language/ control issues involved in "socializing" cyberspace, and the implications for an evolving online architecture of VR products, services, and construction tools.

The Problem. Human reality is profoundly social. If virtual reality is to become a recognizably human place to visit and work in, then it must support people who want to meet there and work together. Facilities for effective multi-user interaction will therefore be crucial to the future evolution of online virtual environments.

Existing online services provide only the most narrow of channels for human interaction: a thin pipeline of text, the occasional bit-mapped image. The growing availability of such higher-bandwidth channels as ISDN has encouraged a variety of alternative developments, including both audio and video links. But these are mostly seen as enhancements to the telephone, rather than as tools for building and operating persistent virtual worlds - their users remain conceptually at their real-world desks.

In contrast, while VR developers are rapidly expanding their ability to model and present complex virtual environments, most of current VR technology is insistently single-user: you can't fit two heads under most helmets! The few examples of (non-immersive) virtual meeting-places capture only a small fraction of the multiplex flow of human social interplay.

This is not surprising, since the tools currently available for building such services address only the simplest level of scene description and object behavior - the Virtual Reality Modelling Language (VRML) being a good example. Having agreed very quickly on a syntax for representing static scenes, its designers now confront the very much more difficult task of extending that syntax to include object behaviors, direct user manipulation, and interaction among multiple user-manipulated "avatars".

The lively discussion over what (not) to include in the next revision(s) of VRML is one indicator of how much information must be captured in any simulation of human social interplay. The sheer hours of talk-show time and pages of magazine space devoted to body talk and fashion statements, spin doctors and image makeovers, should serve as a warning: people have a lot going on when they interact. Human communication is not just multi-threaded, but multi-channel: lots of different (and not necessarily consistent) messages conveyed simultaneously in several media.

That is not to say that cyberspace encounters ought to be photo-realistic proxies for face-to-face meetings. (You don't have to read Dilbert to learn that "one of the best things about the Internet is that nobody knows you're a dog.") But it does suggest that the 'interactivity' issue now hovering on the horizon with VRML 2.0 involves rather more than finding the best syntax for telling an object to alter its relationship to other objects in a virtual world. What we need is a way to model how people relate: their gestures and explicit references, knowledge sharing and negotiation, context recognition and, occasionally, deliberate deception.

I submit that these sociocultural issues are central to the challenge of vr. Enabling people to communicate effectively when they encounter each other in virtual environments - which also means enabling them to capture and re-use the common understandings built up during such encounters - surely this is at the heart of the whole idea of "implementing cyberspace."

Online VR will be a success to the extent that it enables the (co-)evolution of cybercommunities: shared worlds that emerge out of the interplay among the people who meet in them. With that goal firmly in mind, this paper asks 2 questions:

• what do we mean by "human interaction" in VR environments?
• what do we have to design into our virtual worlds, if we want them to support various kinds of interaction?

Agreeing on the answers to these questions - or at least, agreeing what these questions mean, and that they are questions we need to answer - is a first step toward defining the requirements for a "VR toolkit" adequate to the design and construction of genuinely social virtual worlds.

What is an interaction? This is not the first time anyone has tried to answer this question. There is an enormous literature on social action (sociological, anthropological, philosophical), and there is an important line of computer science research which focuses on the role language plays in supporting (and constraining) people's social interaction [cf. W&F86 ]. Nodding gratefully in the direction of these esteemed ancestors (but leaving the detailed bibliography to the graduate students), let us consider the bare bones of what happens when to people engage in any social interaction:

• they identify each other
• they agree on a conversational "mode"
• they pursue their individual purposes
• they break contact, ending the "session."

Each of these interaction phases is accompanied by its own "protocol," in the sense that term is defined in the layered OSI metamodel for machine communication promulgated by the International Standards Organization [ISO]. We will want to consider that model more closely in a moment; but first, consider how moving from real to virtual space complicates these simple procedures.

Recognition/Identification: Since people may choose to represent themselves online by any arbitrary avatar, various issues related to recognition and identification crop up. For one thing, limited bandwidth and CPU-cycles prohibit most representations from looking much like the person represented. For another, many participants will consciously decide to use varying (and perhaps deliberately misleading) representations of themselves. In consequence, the rich set of experience that everybody has with classifying others on the basis of appearance (a.k.a. prejudice) doesn't work anymore. Will we need something like "trusted personality servers" to establish the match between person and representation? More likely, new rules and new behaviors will emerge. As a consequence of the loss of visual clues about a personality, other characteristics will become more important (e.g., voice, conversation style, or interest profiles). It might be sensible to define filters that allow one to specify one's interests and to "highlight" others who match this profile.

Contact protocols. In the real world, the rules for making (or breaking) contact among people are relatively well defined. There are formal protocols for introductions (via shared acquaintances, or "Do you come here often?"), and less formal initiators for specific "talk sessions" among people already in frequent contact ("Have you got a minute?"). The rules of Mail/Forum behavior have been consciously evolved over the past decade ("netiquette"). But translate such forms into VR contexts is not always straightforward. For example, "lurking" is no longer invisible, but as conspicuous - and perhaps as inappropriate - as "holding back" in a therapy group. It would be useful to have a way of gently, nonverbally, signalling our (lack of?) interest...

Interaction protocols. Face-to-face communication is both multi-threaded and multi-channel, with different (and not always consistent) messages being conveyed by body talk and fashion statements, by shifts in the volume, speed and precision of speech, etc. Most of this is lost in current VR settings: your avatar registers very little more than the fact of your presence. We need to provide avatars with modes to match our moods - virtual equivalents for smiling, blushing, etc. Perhaps more importantly, we need ways of managing many-sided conversations: recent work in the CSCW domain could be valuable here (e.g. on multi-user "whiteboards" and support for distributed real-time meetings).

Cultural differences. Meeting people with a very different cultural background was a rare exception until a century ago. High speed mass transport (and especially mass tourism) have made the once fundamental distinction between "my people" and the rest of the world increasingly porous. As world-wide networks begin to bring people from all over the world to meet online, no quick course in "netiquette for newbies" will be sufficient to bridge the cultural gaps across which web surfers will meet and great each other.

Ironically, the much greater bandwidth of VR communication is part of the problem. The thin pipeline of ASCII carries only the bare bones of a message, with little or no cultural meta-information flowing alongside. Under these circumstances, Dogbert may be a thoroughly satisfactory conversation partner. VR will be more nearly like a videophone: we will want to think hard about what background we want to include, and what others can be expected to read into the symbols we use to represent ourselves.

This last point is somewhat counter-intuitive. We tend to assume that increasing bandwidth and providing people with more options ought to make communication easier, not harder. But in fact, adding a new layer of possibility may be like moving from Basic to C++ - with so many additional options, simply saying hello, world! may require a considerable effort.

The OSI metamodel. The main VRML mailing list recently had a long thread devoted to participants' difficulty with this issue of cultural differences. The thread began with the assertion that VR would surely be an contribution to cross-cultural understanding, since it would support traffic in pictures that everyone could understand (after all, everyone can appreciate a sunset...) Gradually, it became clear - and here the contribution of those with multicultural experience was critical - that, while everyone could indeed appreciate a sunset, how each one interpreted the sunset might well be deeply different. What the group was confronting was the difference between syntax and semantics, and thus also between protocols and interfaces: the key distinctions which underlie and unify the Open Systems Interconnection (OSI) model of machine communication (see Fig. 1).

About the Author.