From: jnh@huron.eel.ufl.edu (Jordan Hazen)
Subject: Re: TECH: Are sensory inputs a help or hinderance
Date: 21 Feb 1993 00:52:00 GMT
Organization: UF Electrical Engineering


In article <1m673eINNbpe@shelley.u.washington.edu>, COS.MARSHALL@GLASGOW-POLY.AC.UK writes:
|> 
|> There's been a lot of (interesting) discussion about whether it
|> would be better to by-pass the senses, feeding signals directly to
|> the brain.
|>    I don't know if I'm not looking deep enough into this, but I can't
|> see (no pun intended) what the difference would be, unless of course
|> you have poor sight, hearing, etc. But if all your senses are
|> "reasonably" accurate, then as long as your brain is getting fed
|> the appropriate signals, what difference will it make *where* the
|> signals are coming from. The fact is the brain is receiving, say,
|> visual information so who cares whether its from the eye or a
|> machine (suppose the eye is a machine too). Besides the brain can
|> only distinguish between about 16 million or whatever colours
|> anyway, so even though a machine can provide more, it won't
|> make any difference

I can think of a lot of advantages to using a direct neural interface
rather than a system of physical and sensory hook-ups, once we get the
requisite technology base developed.  For one thing, it might
eventually be possible to intercept and block undesired motor outputs
and (physical) sensory inputs during the course of the VR session.  I
see the most potential in the redirection of attempted muscular
movement-- this should not only eliminate clumsy data gloves and body
suits, but also keep the user's apparent freedom of movement from
being restricted by the constraints of the physical environment s/he
is in.  For example, one would be able to run for great distances or
spin in place forever without worrying about getting tangled up in all
the cables, since your real body wouldn't be moving at all.  I'm not
sure if it would be possible to add additional output channels (to
control extra arms, or whatever), but that's another possibility.

The physical limits on the sensory data sent to the user should be
similarly expanded in a direct-connection system.  Force-feedback
should cease to be a problem-- this could be sent back to the user's
brain along with kinesthetic data (so you can tell where your limbs
are positioned, as in the physical world).  The senses of touch, smell
and taste could presumably be replicated rather easily-- they don't
seem to convey a tremendous bandwidth of information.  These senses
would be almost impossible to realistically stimulate physically
without resorting to some tremendous kludges.

My point is that information is transmitted and processed in the form
of electrical signals by both the human mind and the computer, so why
go through all of the physical problems of sending and receiving data
in physical form? The signals received and transmitted by the brain
are admittedly very different in format from those computers can
easily work with, but wouldn't it be a lot simpler to use a bunch of
demultiplexers & DACs instead of all of the electromechanical hardware
presently employed.

|>   Its like, the ear providing "record" quality input signals, so
|> would there be an improvement if CD-quality sound could be fed
|> directly...
|> 

I'm not sure that that's out of the question-- the 20kHz upper limit
on human hearing is probably more a limitation of the ear than of the
audio processing neural-net in the brain.  The cochlea (sp?) in the
ear sends data back to the brain in parallel form, with dedicated
nerve fibers for different frequency groups.  An analog audio signal
could easily be converted to the proper form using a Fourier
transformation.  I'm not sure if the ear does any further processing
other than spectrum division, but the signals are low enough in
frequency that today's DSPs should easily be able to handle whatever
is required.

Visual input would be orders of magnitude more difficult, because of
the massive number of parallel nerve fibers that run to the brain from
each eye.  The neurons in the retina do some pre-processing before
sending the data on, also.  You certainly don't get a nice bitmapped
image coming in from the eye-- certain simple patterns might trigger
certain nerve firings in the retina, and a lot of the raw image is
"thrown out".  The fact that the placement of individual rods and
cones (density and actual position) is different for each individual
presents another stumbling block. Even neglecting all the
pre-processing that would have to be done, interfacing to the optic
nerves would be a nightmare because of all the connections to be made.
I have a hunch that the image data stream will be the last thing to be
moved to direct neural connection.  We might be stuck with LCDs,
retinal-scanning lasers, and similar imaging systems for a long
time...

I can't see any way of setting up a direct-to-brain connection without
surgically implanting contact wires, processors, etc. within the brain
itself.  To fully replicate real-world stimuli and extract the full
range of motor outputs, it will probably be necessary to target
specific neurons.  Cancelling out input from the senses might be
accomplished without having to cut any existing connections by
connecting one wire to an axon's base and another one a little further
up the line-- a nearby circuit could construct out-of-phase signals
based on what's going into the axon, and insert them at the other end.
I think signals move rather slowly through neural axons, even the ones
with "sheaths", so this shouldn't be too hard.

The brain likes to get all its input in massively parallel form, with
lots of separate input connections (especially audio & video).
Connecting all these lines directly to a computer would be nearly
impossible, so large-scale multiplexing would be needed to send the
huge number of slow-speed signals through one or more high-speed fiber
optic lines (each might carry a few billion bits per second-- the
SCI-FI network standard supports a little over one Gbps).  One cable,
containing the fiber data lines and a couple of power connections for
the implants might be plugged into an implanted sub-mini connector,
without all the mechanical attachments to the body presently needed.

If we're ever able to do this (probably not in the near future), a VR
user should be able to just lie down on a couch, "plug in," and
interact with the environment in every conceivable way without being
constrained by mechanical devices (up to the limits of the computer
itself).  The immersion needn't be absolute-- the same interface might
be employed to overlay the display of a portable PC on one's field of
vision, or to input audio signals from a DAT or CD player without the
bother of headphones (completely private interaction in both cases).