90-05/NASA.2 From: good@baviki.enet.dec.com Subject: McGreevy on James Gibson (long) Date: 10 May 90 12:52:50 GMT Organization: Digital Equipment Corporation Howard Rheingold suggested that I post the following excerpt from a trip report on the ACM SIGGRAPH Symposium on User Interface Software in Banff, Alberta that I wrote up on November 4, 1988. This excerpt describes Michael McGreevy's talk at that conference on October 19, 1988. That talk was my main motivator for starting research in this area - and I think it had a similar effect on at least a couple of others who were present. The focus of this excerpt is to show how James Gibson's theories of ecological optics applies to virtual worlds work. It's basically a paraphrase of McGreevy's interpretation of Gibson, but I'm not aware of anything that Michael has published directly on this topic, so it seems this re-interpretation may be useful. Michael Good Good@Baviki.Enet.Dec.Com ----------------------------------------------------------------- Copyright ) 1988 Digital Equipment Corporation 1.3 3-D focus - Mike McGreevy, NASA-Ames Research Center But now let me describe the undisputed star of the show: Mike McGreevy from the NASA-Ames Research Center. He gave an invited address on "Personal Simulators and Divergence from Realism." I wish this talk had been videotaped - it was one of the most inspirational talks I've seen. My notes capture only a small part of the total effect. Calvin, from the Calvin and Hobbes comic, is the mascot of McGreevy's group. As readers of the strip know, Calvin has a wonderful imagination. McGreevy showed examples from the strip, where Calvin imagines what happens if gravity is arbitrary (he floats away on an adventure) or size is arbitrary (he grows to King Kong size and adopts King Kong behavior). Personal simulators are built into every child. Imagination is always an experimental process. Personal simulators are a tool for imagination in childhood, science, and art. McGreevy's talk then discussed: - varieties of realism, - natural vision: beyond the eyes and brain, - a test of utilitarian realism, - remapping the physical world, and - illustrations of work at NASA-Ames. 1.3.1 Varieties of Realism Most graphic systems today view the user as a pinhole-camera; or, if they are stereoscopic systems, as 2 pinhole-cameras. This is not an very enlightened view for providing realistic graphics. In most systems, the image is not related to the user. Images are constructed as viewed from a single point; if the user is not at that point, the image lacks realism. A more realistic interface would also stimulate peripheral vision, not just focal vision - it would be wide-angle, not just 400 of angle. There are spatial and dynamic links to the user ignored in current systems. 1.3.2 Natural Vision McGreevy's discussion of natural vision was based on Gibson's "Ecological Approach to Visual Perception." This theory was new to me (as are most things in visual perception), but seems well worth exploring as it appears to come from a similar perspective to our own work, and perhaps to Maturana's work in vision as well. In this approach, natural vision is viewed as part of our body, not as a separate phenomenon. A comprehensive visual interface must enable the user to directly and naturally observe and manipulate objects and explore environments. To perceive the world is to co-perceive one's self. We perceive meaning and utility of the environment and objects within it by observing our own capacity for visual, manipulative, and locomotor interaction with the environments and objects. Environments provide a context; objects provide operations like grasping, manipulation, and much more. Current graphical interfaces fall far short of these goals. Current interfaces are sterile: unlike physical environments, they have no indigenous creatures, no objects with a history that reflects experience, and provide poor representations of other people within the environment. For manipulation, a mouse for examples severely attenuates what people can gesture to a system. 1.3.3 A Test of Utilitarian Realism This discussion introduced Gibson's concept of affordances. Affordances are reflected in the following test of utilitarian realism for artifical realities: Are the objects and environments as real as I am, and do they afford me the opportunities for use, interaction, manipulation, and exploration that they appear to offer. For example, many graphics system use wine glasses to illustrate their capabilities. Well, does the virtual wine glass hold virtual wine? Can I drink the virtual wine? Can I throw it in the fireplace and have it shatter? In another example, consider an architectural 3-D system. Do walls block my progress, or do I just fly through them? 1.3.4 Remapping the Physical World The realism of affordances serves scientific visualization. Remapping the inaccessible components of the physical world into virtual objects and enviroments supports exploration, experimentation, and imagination. Earlier examples from history show other types of visualization. For example, medieval and renaissance art was often a form of religious visualization. McGreevy showed an example of a church ceiling, which when viewed from the proper perspective on the floor, makes it appear that the church is opening up into the heavens (for example, columns in the church are lined up with columns in the ceiling painting from this perspective). Perspective drawings like this are distorted, sometimes very severely, when not viewed from the same point from which they were drawn. In this case the distortion causes the religious visualization to break down. But this type of distortion happens very frequently in current graphic systems which treat the user like a pinhole camera. 1.3.5 Work at NASA-Ames McGreevy then gave an overview of three generations of head-mounted displays developed at NASA-Ames. These displays use technologies such as wide-angle optics, a pair of LCD displays with LED arrays, and head trackers. Though higher-resolution LCD's are coming, a pair with 100x100 resolution worked amazingly well in early 1985, since you could move around and see things from different perspectives. Now, sound has been added with headphones and a microphone, and more work has been done on the affordances of the interface. It's important to perceive the floor at your feet. 3-D escalators should not have their steps hit you in the chest. The first version of the head-mounted display used a red, white, and blue motorcycle helmet painted with a NASA logo. This version is still often used in NASA photos for its photogenic quality. However, later versions are less claustrophobic. One reason is that you can get a sea-sickness effect with this display, I imagine especially in people who are prone to motion sickness. Not having a helmet helps alleviate the problem somewhat, and works better in cases where the user does get sick. They have used visual environments to explore physical environments, in the context of planetary exploration. McGreevy showed two videotapes of work done with JPL. The first was a visual environment of Los Angeles. Using techniques developed in that project, they then developed a visual environment for Miranda, one of the moons of Uranus photographed by Voyager 2, and we saw "Miranda: The Movie". The head-mounted display work is described in the NASA Tech Briefs of July/August 1988 (Vol. 12, No. 7). NASA is transferring this technology to anyone interested in it... The question-and-answer session had some interesting moments. Bill Buxton asked about doing work in real artificiality as well as artifical reality. McGreevy agreed, saying that's what Calvin does in the comic strip. One person asked about artists working in this area. McGreevy pointed to the work of Myron Kreuger of Videoplace fame. Someone asked about using holograms, referring to Steve Benton's work at the MIT Media Lab. McGreevy's reply was that with head-tracker technology working now, there was to need to wait for technology like holograms. Another person asked about incorporating tactile feedback. McGreevy replied that they've done a little bit of work there and see a movement to the use of exo-skeletons. In conclusion, McGreevy reiterated his call to go beyond a pinhole model of the eyes, and called for work on personal simulators to provide another step along the ascent of humanity.