From: Jeremy Lee Subject: TECH: My standard is better than your standard II Date: Sun, 28 Jun 92 04:24:06 EST It's not often I reply to my own posts, but I after a little discussion and thinking I decided to add a few things. Please keep discussing this with me. It's going to form the basis of a VR project, so I'd like to get my initial concepts hammered out. > Decided to throw my ha'pneys worth in. > > Basically, it doesn't matter how you implement a VR system. You can talk > about graphics systems standard 'till you're blue in the face, and > no-one will follow them because they all think that they can code a > better raster package than you. > > What you want to do is define a way for object databases to be stored, > to describe a VR 'scene', and the communications protocol necessary to > make sure that everybody gets concurrent access to this database, and > gets told about updates when they happen. > > Obviously, even this is difficult, becuase the type of data storage > system you use also reflects in how you write your graphics system. > > I, like everybody else, have a personal dream about how to do VR. I'll > explain it here, and leave it open to discussion. I am thinking about > this *very* seriously, becuase we are trying to get the stuff together > here at Bond Uni to set up a proper VR Lab. (By proper, we mean more > than datagloves and amiga's. For instance, a couple of Indigo's would't > go astray, nor would a mainframe or two, if anyone has CPU time they > would like to donate to some real research.) > > Let's take it from the front back. I'll begin with my preferred way of > doing the graphics front end, not for it to become a standard (I know it > wont.) but because I think ideas should be shared, and It also infringes > on the way I would like to set up the underlying system. > > First, the basic graphics primitive is the patch. A patch may be > anything, depending on the complexity of your system, from a simple > triangular facet to a bicubic spline patch. And many things beyond that. How about a volume rendered patch, or a fractal one. > Each patch describes how to render itself, right up to providing code. > This code may simply call 'standard libraries' that are common between > platforms, but it will still hold. For a demonstration of this, take a look at the RenderMan interface. You describe each object as a little bit of code. Renderman doens't go as far as I would like, though. As well as describing new textures I want to be able to define entirely new graphics primitives. But, all this has to be encapsulated. Your renderer doesn't have to be able to handle the patch before connecting to the VR, because it is intrcuted how to render it when the object data comes down the line. Everyone will have the same basic renderer, and upgrades are not necessary, because your software get's "upgraded" to handle the database every time it logs in. > You have objects defined in terms of groups of patches. Each object is > also an active thing, and basically contains code that defines how the > object acts when things happen to it. Not that this is _close_ to an OO > model, but not quite. Things that happen include messages from other > objects, and interactions with it's patches. The distinction is subtle. > What I am proposing is that the environment does not 'move' an object > when it interacts with another, but that it notifies it of the fact and > the object moves itself. Or not, depending on how it feels. And I repeat, although this is _close_, it is NOT an OO model. There is no inheritance. Presently I have no concept of security, past being able to connect to the DB in the first place. Although each object is encapsulated, it is not isolated, and any other object is allowed to change it. There is no need for the concept of "parents" or "friends". These concepts, applied to a VR, are very alien. I have used OO 3D editors and they are unintuitive. Inherited methods are almost useless, as each object is a seperate, individual object. It is encapsulated and complete in itself. This is my fundamental proposition. The object has to continue to exist if the parent object is deleted. If you want to move an object to another database, then why should you have to move it's parent? OO design is a static thing. VR is a dynamic environment. When I ask why OO is wanted, most people indicate a need to reduce the trouble when *creating* the object. If this is the case, then copy a "parent" object and redefine some scripts. If you want attributes of another object, then copy the scripts from inside that one. I am not saying that an OO language is appropriate for programming the environment, or even the script language, I am saying that OO not not suitable for the VR environment itself! > Imagine this. Another patch intersects with one of the patches of the > current object. This fact is detected by the environment, and both > objects are notified of all the relevant details. What happens next is > up to the individual objects. They may decide to "converse" for a little > and decide upon a course of action. (message passing between objects is > allowed. Described later.) The most likely thing, of course, it that > they will decide to 'move'. The environment will have already notified > the two objects of their relative vectors, surface normals, and > intersection point. A normal object will do a little straight > computation, and decide that it now wants to be moving in a new vector > with a different spin. It may decide to change it's shape. It may decide > to change it's colour. It may decide to emit a noise. This is all up to > the object. It may choose to ignore the message entirey, and will > continue to sit rock-steady as the other object 'bounces' off it. > (Decidedly non-newtonian.) You may cry "What about consistency!". Well, > you will just have to metaphorically grab that particular object by the > scruff of it's metaphysical neck and tell it to do that in future. > Obviously this is a very powerfull way to describe objects. I have noticed a problem with this. If you can redefine the basic geometrey of a patch between objects, then how do you get the environment to tell when two patches have intersected? You can't, unless patches are based on some sort of basic geometric model. Patch collisions revert to a (very complex) mathematical operation that tries to detect when two 3D manifolds intersect. If one is a fractal manifold, then this can be difficult. How do you do t without limiting the types of patches you can support? The only idea that I can come up with is setting crude "bounding boxes" around the objects for efficiency, and when the environment detects the intersection of these bounding solids, then it should get the two objects to talk to each other and find out whether an intersection has occured. This of course is difficult when one patch may be using a CSG model, and the other a fractal descriptor, or possibly a bicubic patch. I have the vauge glimmering of an algorithm, but would rather not investigate it because it seems horribly computer-intensive. any solutions? > Back to message passing. Imagine our hypothetical collision. The two > objects may want to pass a message. Passing a message basically involves > sending some encapsulated machine-readable message, and not hoping for a > reply. Once again, it it up to the recieving object to accept or ignore > the incoming message and decide upon what to do with it. Obviously, you > may want to lay down some sort of initial 'Rules of Object Etiquette' > that it would be considered rude to violate. Rules such as inertia. When > two objects collide, they tell each other about their current mass, > which will help to compute what happens next. Objects may request > information about the other object, but needn't expect a reply. Some of these "Rules of Object Etiquette" may include allowing one object to tell the other where to move. Remote manipulation is one application, and the other is when your "hand object" has graped something. If we rely on the traditional geometry, then the grasped object may move in a very jittery way if it has to simply respond to the forces that the fingers of the hand are applying, but we can code a script in the hand object to detect when two or more intersections have been made with an object, and if some "grasping" parameters are satified, then instruct the object to smoothly follow the motion of the hand, and not to go where it was planning. Notice that the script based solution allows some objects to omit this particular event. All this means is that our hand will simply pass stright through the object, like a hologram. I'm sure that you can imagine when either behavior, or something even stranger, would be called for. In a games environment, many objects would have a "self-destruct" script. A "bullet" hits something, and asks it to blow up before doing the same thing itself. (or the object may decide to accumulate damage, or alter it's shape dependant of the energy of the projectile) > Each object, as mentioned, has it's own script defined, in whatever > language you are going to define. (more on that later.) These scripts > are event-based. An event happens, and the appropriate "handler" in the > object gets called. "Events" can also be requested by the object, such > as timers going off, or other things that the Environment may provide. > There is also the concept of the "null event" which gets called when > nothing is happening to the object. The closest language I have used > that is like this is Hypertalk, and I think that it would actually be a > very good basis for any VR language. These scripts are defined to be concurrent. Any or all of these events may happen at once. Resource clashes may occur inside the object, but that is for the "builder" to worry about. Once a working "object" is built, then you only need to copy it and make minor modifications to make a new object. Imagine the constant changes that would occur in the children in an OO heirarchy. I can see only a need for linking inside a VR, not inheritance, and that can be performed with messages. Each object could very well have it's own "main script" that is constantly running, providing updates to it's position and rotation. As an aside, I would challenge anyone to produce a situation where and OO design would work and the above scheme would not. > You have objects, which are composed of patches (the "physical" bits) > and event handlers. (The "logical" bits.) The two obviously must be able > to affect each other. Things happening to patches cause events, and the > event handlers may alter/add/delete patches. Also, event handlers (or > their assosciated subroutines) must also be capable of > adding/modifying/deleting event handlers too. This makes each object a complete encapsulated entity. Each object can exist by itself, and change itself. Each object can also pass messages to other objects, or directly modify those objects. Of course, it must have come in "contact" with those other objects at some time. The way to imagine each object is to think of it as a seperate computer, connected to a network. It passes messages, and tells other things where it is, and can be sent code from other machines, but is basically an independant unit and does what it wants. There is presently no concept of a "God" object which omnisciently knows everything about the scene, and which objects can enquire of for information. I do not think this is necessary. The "Environment" has as little power as I can make, and as much is relegated to the individual objects as possible. All "The Environment" does in the model so far is detect patch collisions and send messages to the objects. Even this can be scrapped, if we want to let each object take care of it's bounding solids. (Hmm, an algorithm to detect collisions has sprung fully-fledged into my head now, and it's even efficient!) Each object computes it's own bounding solid. It checks for an intersection with the bounding solid of any other object. If it does, then it calculates the intersection solid. Each object then computes the bounding boxes (multiple) that describe the disconnected parts of it's surface that extend into the previous intersection. The other object does the same. Each object now has a set of bounding boxes, and more tests are done to see if they intersect. The whole operation is done recursivley until (a) no intersection is found (the latest intersection volume is empty of either object) or (b) an intersection bounding box is reached of sufficiently small size to be indistinguishable from a point. All (all he says!) the individual objects have to do it compute their own bounding boxes, and tell their neighbors via messages. Aack! How do they know who their neighbours are? Forgot about that. Either we have to have a "God", or the enviroment has to watch intersecting bounding boxes. The more efficient, and pleasing solution would be to let the environment do it. But, we have reduced it down to approximatley 5 lines of code. > This brings us to the next problem, how to identify the objects and > patches in the system. I don't know, but obviously we need a good > method. Any suggestions? Objects are also allowed to refer and send > messages to objects encountered in the past, which it may have stored > the reference too. After a little thinking, I've decided that it should have both a textual name, for human consumption, which can be non-unique, and some incredible 256bit number which is garunteed to be unique. Perhaps a varaible length number to allow for future expansion. The problem is that I think every object in the world (as in, the ENTIRE world) should be garunteed to have a unique number, so that when messages get sent, in an internet-like network, it will eventually find it's way to the target object. Like site numbers in a way. > What event handlers should each object incorporate? That's up to the > person/agent/object that last fiddled with the object. You have the > concept of 'injecting' and 'extracting' event handlers from an object. > (I can imagine a VR tool that looks rather like a Hypodermic needle - > just stick it into the object and it adds a new event handler! ;-) Of course you are going to have to pull yourself up by the bootstraps by writing a few scripts outside the VR environment, but once a basic set of tools have been established, all further work can be done inside, including writing the scripts and defining the patches. How to type inside a VR environment? I have a REALLY good idea for that one. > Now, what language are we going to write this in? Don't know. We are, > however, going to have to make a new one. It's not going to be C, or > C++, or pascal, or Lisp, or bloody COBOL. None of these are adequate, > and I'm not going to argue on this point! A VR environment will require > a human/machine readable language that is self-modifying, and > _very_high_level_. Look at Hypertalk for a good idea, but it's also > going to incorporate a lot of other ideas, such as a database managament > system. It's also going to be interpreted, because don't forget that you > have no idea on what machine (or machines) it's going to be running on, > and it will be constantly changing itself, maybe. I know that > interpreted languages are slower (they can still be pretty quick, if > they are sufficiently advanced.) but computing power will keep > increacing, and I think that we realise that VR is going to eventually > be implemented on BIG computers. Note that these scripts will be > executed by whatever machine is running that environment system, so you > can have a lovely big cray acting as a dial-up server with your little > workstation taking care of the graphics. The server need only notify you > of what happens to the relative position of the object, and how the > geometry of it's patches have changed. PLEASE PLEASE DO NOT USE C OR ANY VARIANT. C suffered immesurably when it was made into C++. C is a lovely systems programming language, but Ritchie would turn in his grave (sorry, he's not dead yet, is he.) if he saw it prostituted to the degree that would be necessary here. We need a 4GL at *least* here. I personally favour some hybrid of FORTH and HyperTalk. And remember that the scripts might get shifted, mid-execution, to another machine. Also remember that the scripts must be able to modify themselves, And other objects, in very high level ways. You cannot achieve this with mere libraries, no matter how overloaded your operators. I doubt that you can achieve this with a compiled language. Defining this language, given the broad constraints I have set, is not going to be an easy task, and I would much rather not do it. I don't know that much about compilers and languages, and it's going to be a horrendous task. All I can really suggest is that we define a language that can define itself (which is why I suggested FORTH) and so each script is really it's own language, interpreter, and program. (Just a thought....) > The renderer gets told to begin with (when you connect to the database) > everything that you need to know (and your system tells the server how > much you need to know.) about the patches. This includes downloading the > texture maps, and the code that renders the marble. You can even go as > far as including the code to render it right onto the screen. In future > systems, I can see the system asking each object if it actually appears > at each screen co-ordinate. Imagine this, the system distributes a > message saying "OK, guys, I'm now computing this pixel, and I want y'all > to tell me if you appear at it, and if so, at what distance from the > observer, and if so, what pixel value to use, and what transparency." > Each object then reports back RGB triplets and a trasparency value. Yes, > I know, this takes a lot of power. Yes, I know, we are not currently > using crays. But one day we will. Extending my thoughts about the intersection tests, it occurs to me that a similar type of CSG renderer would be appropriate. I recall seeing something about an IBM (mainframe, that is) program called WINSOM. A camera object asks each for it's bounding box, and recursivly divides down the the pixel level, where it asks for a pixel value/transparency. I'm trying to think of a way to get optical effects like refraction & reflection & specular highlights in there as well, which is where WINSOM comes in. On top of this, I'm trying to think of a way that you can implement the renderer as a scaleable architecture. > The way I would like to implement the environment is with a radiosity > renderer included, so that each object computes it's form factors etc > and takes a hand in computing it's own little set of simultaneous > equations, and then modifies it's patches. All you have to do is to > remove this event handler from various objects and they don't do it. A > radiosity method is essential in my eyes, because it generates a > directionless lit model of the scene. This is still just message passing between the objects, but it's hard to imagine the existance of such a scheme without a "God". I'll have to re-think that. > When you want to put it on the screen, then you do what you want. You > don't have to take any notice of the radiosity values if you don't want. > In almost all cases, the object database will contain more information > than you currently need. In case you hadn't noticed, the database is a very active thing. Rendering the scene will actually take a fraction of the processing power that the object interaction will take. This is as it should be. Maintaining the world independant of whoever is viewing it. > We also need to take a look at distributed processing. If you hae a > number of machines on the network, then I don't see any reason why the > master databse manager shouldn't ask a lightly loaded machine to "take > care" of one or more of the objects. Obviously the client will have to > instantly report back any changes to the object, in case the client goes > down or the connection breaks, but the client no longer needs to request > information about that particular object (since it's all local) which > saves a little bandwidth to help compensate for the increace that having > to handle that object will take. If you are running a user-pays system, > then some users may actually be paid money for the contribution that > their system makes! > > Handling an object will basically involve managing that objects events > by running it's scripts when the master database manager gives it > events. If you can de-centralise this system, then you have opened the > door to truly massive parallel processing. This may mean not having a > central manager, and if a node goes down, then the object just > dissapears. This isn't nice, and you want a backup somewhere. I'm not so sure about that now. I think that you will just have to accept that as an inevitable part of virtual life. If a node goes down, then tough. You will just have to wait until it comes back up. Parallel processing is most definitley the way to go. Even if the system is implemented on a serial computer, then it's going to have to emulate a parallel system, as each script in each object runs concurrently. And, what happens if your script is a parallel language like OCCAM? Attach each object, each script even, to a transputer. Message passing between objects is equivelant to data channels between chips. Want a bigger system, capable of handling more objects? Just add more transputers. > Where does the observer fit into all of this? They are just another > object, handled by the local machine. In this case, there is also a link > between the physical pheriperals of the machine and the 'body objects' > defined in the computer. With this model, when you wiggle your little > finger, what happens is that the message gets sent, as an event, to the > 'hand object' inside the environment. If you have defined the event > handler correctly, then the little finger should wiggle. and don't forget that since it is a script, then it can be re-defined inside the VR, so that wiggling your little finger might shake your left leg. Makes a lot of the things Lanier talks about possible. > This then reduces the entire system to three components. The IO system, > the renderer, and the object handlers. I'll redefine this. The IO system, the renderer, and the Object database. The Object Databse is of course the biggest part, comprising of: Message passing system Script handler Object handler An Environment (handles object interaction) A God? (Omniscient central source of knowledge.) God will of course answer questions that the objects ask of it, such as "What other objects are in this scene." "Where did XXXX go?". It would also be the part that tells the renderer which objects to render, and where they were. The Environment and God are of course central objects, and I would like to get rid of them if I can. ("That's rather like old Neitzche, isn't it?" piped up Heidegger.) Outside of this will be machine implementation details such a a parallel distributor. > Nothing of what I have described here is new. It's a logical > alternative. It's current disadvantage is, however, that it will be slow > on most current systems, and very difficult to run on low end machines. > However, its expandable and flexible. These are the two traits that are > needed in a world where computing power is growing yearly. In ten years, > we will each be able to buy the equivelant of an IRIS workstation for > our kids to play around with. That is when VR will take off, and we need > a system for the future, not for today. The harware isn't powerfull > enough yet, guys, but we can still define a good standard that will be > usefull when it does arrive. I disagree with the view that we have to get VR out into the marketplace NOW. I think that VR will benefit from staying inside academia for another few years. Like UNIX, the pure rarefied atmosphere of university will cause VR to take on a pleasant, robust shape. Ten foot tall and bulletproof, that's my motto. Commercial applications have traditionally been the last, and for a reason. *********************************************************************** * . Jeremy Lee s047@sand.sics.bu.oz.au Student of Everything * * /| "Where the naked spotless intelect is without * * /_| center or circumference. Look to the light, * * / |rchimedes Leland, look to the light" - Dale Cooper * ***********************************************************************