From: brucec@phoebus.labs.tek.com (Bruce Cohen) Subject: Re: half silvered lenses (was Re: Direct Neural Input (Was Re: Date: 16 Nov 91 01:49:29 GMT Organization: Computer Research Lab, Tektronix Inc. In article <1991Nov14.052546.21344@watserv1.waterloo.edu> dstamp@watserv1.waterloo.edu (Dave Stampe-Psy+Eng) writes: > It seems the brain keeps track of "adjusted relative times" between > internal and external events, and changing the internal/external delay > upsets it. In effect, we're all living a good part of a second in the > past... As I understand the current thinking, keeping track of those delays is necessary to a lot of perceptual processing. I seem to remember hearing recently there's some data now to show that dyslexia and its aural counterpart are caused by unequal or perhaps unequally compensated-for delays in the left and right hemisphere pathways to the central word recognizing areas. > brucec@phoebus.labs.tek.com (Bruce Cohen) writes: > >>For many tasks, I suspect that less than 10 frames per second will do, >>possibly as little as 6-7. The positive feedback and loss of >>coordination effect seems to have a sharp onset somewhere in this range. >>But the onset does seem to depend on the task. Someone want to provide >>some hard numbers? All I have are some informal experiments I did to >>prove to some doubting Thomases that less than 10 frames was good enough >>sometimes (they wanted me to rip apart a year's worth of work because >>they were convinced that less than 20 would be unacceptable). > > I suspect it depends strongly on a) the amount of blurring in the motion > and b) the speed of the motion itself. Does this fit? Yes, I think so. The experiments I worked on didn't use any blurring, so I can't speak to that, but the onset rate of coordination problems seemed to depend to some extent on the amount of motion required by the user: jerking the controls hard caused positive feedback at higher frame rates. This is what I would expect if the control system model is the domininant cause of the coordination loss. >>Personally, I'd rather have the virtual world overlay the real one; I'd >>be more confident of not tripping over my chair (no smilies AT ALL! >>I did that once while trying to videotape an event with a minicam. >>Restricting or eliminating your view of the real-world while wandering >>around in it is likely to be very rough on the ankles if nothing else.) > > Excellent point! The only other way to prevent this is to either stay > seated (virtual desk) or have a special VR room (which might be OK, as > it could be thoroughly instrumented thus allowing better body tracking ). You'd also have to work the position of the walls into the model the user was navigating, else she'd bump into them. But this does raise another question: how much can you distort the user's model of the room geometry from the actual by subtly changing angles and distances, forcing perspective, etc. There are two possible effects you could use here: 1) being able to render the scene in real-time allows you to continually modify an impossible image so that it seems impossible from all angles (consider constantly re-rendering a Penrose trine so that is looks real from the current viewpoint) 2) take advantage of the low-frequency cutoff in the response of the orientation proprioceptors like the balance organs in the inner ear. If we slowly change the image as the user moves a little more or less than the movement calls for, we could make a circular path around the room seem like a long straight-line journey. This is the sort of effect that I was referring to in a previous message about the use of theatrical effects in designing worlds. > Problem is know what is being looked at.. the general region of the > screen will have to do, and even then you need a $20K+ gaze-tracking > system. My experience (building and using) these shows that the > correspondence between gaze position and attention can be VERY loose > indeed. Seems to be more of a link between required visual acuity > and gaze offset from the object (unproven, but I've got some research > planned to check this out). OK, how about a compromise system for the near-term? Let's plan on using a headset, and build a cheap eye-tracker into it. Seems to me the design problems for an eye tracker which remains a few centimeters from the eyes and always in the same orientation to the head are much simpler than the $20,000 ones, so it ought to be cheap. You'e need two, of course, but there ought to be a lot of common subsystems, both optical and electronic. > The important word is BLURRED. For any object moving at less than > 30 visual degrees/sec, the eyes can track it with very small errors > (less than 1 degree, or 3 pixels with current VR systems). Thus > eye movements are far more important than head movements for saving > drawing bandwidth. Of couse, usually head movments correspond to > attention shifts, but slow ones may be tracking movements. Maybe > blurring during head movements will force the user to keep his head > movements slow and infrequent. It seems to me that you could distinguish the ballistic motions associated with attention shifts from tracking motions by the jerk (rate of change of acceleration) at the start of the motion. Ballistic motions have a high jerk at the beginning of motion, followed by zero jerk and acceleration. I'll bet that tracking motions tend to be smoother, with a near constant jerk for a longer period of time than ballistics. > BTW, since rendering "blurred" edges can be a bit costly, what about > a "postprocessor" on the video output that deliberately blurs specified > picture areas? Very simple if you are using a CRT display: just change > the focus voltage in the monitor. I like the postprocessor idea, but I'd rather do 2D digital filtering on the framebuffer output because it would allow filtering for variable resolution, if we can come up with a good way to aim the display at the foveal area. > Predicting where a saccade will land for hi-res area drawing is not > not very accurate, and you need a FAST eye-tracking device (>50K$) > to get acceleration/velocity parameters on the saccade. See above. Cheap DSP chips ought to help with this. > I predict > that the blurring extends up to 50 mS past the end of a saccade, due > to the relatively long response time of retinal receptors. Thus > it's probably OK to not predict the saccade, if your renderer is fast > enough. But you could still get some advantage if you can compute the motionless background objects to higher resolution than would otherwise be practical in that 50 ms deadtime. ------------------------------------------------------------------------ Speaker-to-managers, aka Bruce Cohen, Computer Research Lab email: brucec@crl.labs.tek.com Tektronix Laboratories, Tektronix, Inc. phone: (503)627-5241 M/S 50-662, P.O. Box 500, Beaverton, OR 97077