Subject: Linux-Development Digest #817
From: Digestifier <Linux-Development-Request@senator-bedfellow.MIT.EDU>
To: Linux-Development@senator-bedfellow.MIT.EDU
Reply-To: Linux-Development@senator-bedfellow.MIT.EDU
Date:     Sat, 11 Jun 94 13:13:05 EDT

Linux-Development Digest #817, Volume #1         Sat, 11 Jun 94 13:13:05 EDT

Contents:
  Mach64 Drivers (jdrumm on BIX)
  Re: Filesystem semantics protecting meta data ... and users data (Totally Lost)
  Whats changed in the serial dirver ? (Ernie Elu,,015-176-001)
  Re: SMP supprot for Linux???? (Elaine Walton)
  Buslogic 445S driver (Bill Heiser)
  Re: Linux game development (Was Re: Why [DOS, W (Kevin Brown)
  Is there a list of all Linux devicedrivers??? (\rjan Derel|v)
  Re: My problem? GCC problem? Linux problem? (James Logajan)
  Re: Filesystem semantics protecting meta data ... and users data (Totally Lost)

----------------------------------------------------------------------------

From: jdrumm@BIX.com (jdrumm on BIX)
Subject: Mach64 Drivers
Date: 10 Jun 94 00:32:28 GMT



Is any work being done on an accelerated X-Server for the new
ATI Mach64 chip?

------------------------------

Crossposted-To: comp.benchmarks,comp.sys.sun.admin,comp.security.unix
From: idletime@netcom.com (Totally Lost)
Subject: Re: Filesystem semantics protecting meta data ... and users data
Date: Sat, 11 Jun 1994 14:43:14 GMT

In article <Cr7IEu.Epw@ucdavis.edu>,
Kevin Brown <kevin@frobozz.sccsi.com> wrote:
>In article <idletimeCr7F84.8FF@netcom.com>,
>have the expertise and the ability.
>
>So why don't *you* write an efficient, secure filesystem for Linux
>(or one of the free versions of BSD, if that suits you better,
>though they may be a bit more entrenched in tradition than the
>Linux developers)?  You have the source code, so you can make any
>changes you need to in the interfaces, drivers, etc. to make it
>happen.  You might even be able to work with the people who wrote
>the device drivers.
>

I am self employeed, and run a small consulting company with
a house and kids to support (IE current fixed expense >5K/mo).
My short term goal is to save enough to return to grad school soon.

This is not the only critical flaw of UFS or other V6/V7/SvRx derived
filesystems. To do the job right is a ground up redesign of a
filesystem, BIO, Kernel Interfaces, drivers, and supporting utilities.
It would take myself and two/three junior helpers probably a year to
complete to production release status.

I am not ready to starve for that long and sacrifice my house and
kids (and education) to put a work this size into the public domain.
I'd put the core work into a raid controller and attempt to make
a product out of it first.  Any other path requires a system
vendor willing to make major changes to the base OS and porting
all drivers and other filesystems (at SCO this is a BIG DEAL)
AND making life tough for a bunch of third parties that will have
to do the same. When I was at SCO they had just introduced SCO/UNIX
and had forced much of the XENIX customer base to make the big
change with them. Every five years is  about as often as a
major vendor can afford this size wrinkle to customers.

After SCO I discussed this with both NCR and Compaq
to see if major functionality improvements would be
an incentive to customize the SCO product. SCO is
a STANDARD in their market ... and not to be fiddled
with. Given the NIH factor and no existing relationship
I didn't see any practical way to get DEC, Sun, or USL
to fund the project. I don't think there are any other
UNIX vendors left besides these that could afford the
pay for it.

The only other long shot is to do the work in FreeBSD
or the lite when release and make a product out of it.  I
would have to think long an hard about that, and have been.

>Am I right that a priority-based buffer cache would be sufficient
>to get the characteristics you need for the reliability and security
>requirements?

as I discussed elsewhere a simple priority approach doesn't
quite work ... 

John

------------------------------

From: ernie@tinny.apana.org.au (Ernie Elu,,015-176-001)
Subject: Whats changed in the serial dirver ?
Date: 11 Jun 1994 20:22:26 +1000


For a couple of weeks now I have been trying to get my AST fourport running
with kernel versions later than 1.1.13.
I am now trying with kernel 1.1.18 again. I have noticed you can no longer
define the AST fourport in serial.c, after posting to the net it was
suggested that I try the following :

in serial.c
#define CONFIG_AUTO_IRQ --------------> add this in source file

modify this line (the line with #define FOURPORT_FLAGS) to look:

#define FOURPORT_FLAGS (ASYNC_BOOT_AUTOCONF | ASYNC_FOURPORT |
ASYNC_AUTO_IRQ)


I have done this and the board is now recognized on boot up however the
kernel says it is set to irq 0 instead of 5 which it should be.

Serial driver version 4.00 with no serial options enabled
tty00 at 0x03f8 (irq = 4) is a 16450
tty01 at 0x02f8 (irq = 3) is a 16450
tty08 FourPort at 0x02a0 (irq = 0) is a 16450
tty09 FourPort at 0x02a8 (irq = 0) is a 16450
tty10 FourPort at 0x02b0 (irq = 0) is a 16450
tty11 FourPort at 0x02b8 (irq = 0) is a 8250

What is the correct syntax for the AST FOURPORT in serial.c ?
Is there a posting covering the changes to serial.c over the last few
revisions ?

- Ernie.
-- 
_______________________________________________________________________________

    Australian Public Access Network Association  - ernie@tinny.apana.org.au
_______________________________________________________________________________

------------------------------

From: ewalton@magnus.acs.ohio-state.edu (Elaine Walton)
Subject: Re: SMP supprot for Linux????
Date: 10 Jun 1994 01:22:45 GMT

Using News, you can search for certain topics.  Read your help in your news-
manager.
At present, there has been a lot of talk on the subject--not much done.
The problem stems from the sheer problem of a highly-coupled MPU (multi-
processor unit).  Some have suggested porting Linux over to the Transputer
processors to take advantage of it loosely coupled arrays.  For the most
part, a MPU would require restructuring the OS and the compilers (unless
you can manage the critical sectioning yourself--not an easy task;).
-Sean

------------------------------

From: bill@bhhome.ci.net (Bill Heiser)
Subject: Buslogic 445S driver
Date: 11 Jun 1994 16:23:44 GMT

I am replacing an Adaptec 1542b with a Buslogic 445S so I can take
advantage of my shiny new VL bus :-)

Which driver is best for me to use?  Is the new Buslogic driver "safe"
yet?  i.e. is it causing anyone any filesystem problems?  Should I just
(continue to) use the Adaptec driver for now?

My buslogic card says it's a "Rev C".  (I've seen some recent posting
concerning Rev D I think).  Should I contact Buslogic for an upgrade?
Maybe this was old stock? (it came from Hard Drives Intl).

-- 
Bill Heiser:    bill@bhhome.ci.net,  heiser@world.std.com

------------------------------

From: kevin@frobozz.sccsi.com (Kevin Brown)
Subject: Re: Linux game development (Was Re: Why [DOS, W
Reply-To: kevin@frobozz.sccsi.com (Kevin Brown)
Date: Thu, 9 Jun 1994 22:33:14 GMT

In article <JAYHAN.94Jun8112203@derkins.cs.washington.edu>,
Jay "Thierry" Han <jayhan@cs.washington.edu> wrote:
>In article <7028@raven.ukc.ac.uk> gjh@ukc.ac.uk (Greg Harewood) writes:
>
>> With no other processes running, Linux is *fast*. You have a flat memory
>> model with as much space as you want given virtual memory. You have fast
>> efficient drivers for all your hardware. It evens things out to a much
>> nicer situation where all machines has as much memory as you want, a maths
>> coprocessor, and the only difference between platforms becomes speed of
>> execution. And the multitasking environment means you can have a process
>> for the user interface and another to render the graphics (for example)
>> in as much detail as remaining processor speed will allow.
>
>Ah, but how do you know how much processor power you have?  That's
>one of the problems of using UNIX as a real-time OS.  It's also a
>pretty open research field to figure out what kind of interface
>would be needed between application and OS so that they cooperate in
>sharing the processor.

In general, the OS takes however much time it needs to take in order to
get the job done.  What you're asking for here is basically a real-time
operating system.  Unixoid real-time operating systems do exist, of
course, but games are rarely so dependent on real-time constraints
that they require such an operating system.

As for how much processor power you have, that's what BogoMIPS is for.
:-)

>There are problems with UNIX for writing efficient stuff.  For
>example, the virtual memory system.  If you have to page in memory
>that you thought was there, your timing goes to hell.  

But the thing you're overlooking is that if you have to page in
memory at all, then for the purposes of running your program you
probably don't have enough memory!  I admit, the concerns you have
here are much more relevant when you have a bunch of other processes
running, but for games this isn't much of a concern.  Simply tell
the user in the instructions that they have to run your game on an
unloaded machine or the performance will suffer.

A possible help to this might be a system call that basically says
"lock the pages that contain the range of addresses between X and Y
into memory".  Doing this, of course, will increase permanent core
memory use, and more paging for *other* things will happen as a
result.  That tradeoff will have to be considered.

Ultimately, you'll have to do the same thing that all other game
manufacturers do: "This game requires a 386DX-40 or better with
at least 4 meg of RAM, running someOS version whatever, etc."

>That's why
>many databases use pinned memory segments that they manage directly.
>Also, device drivers are good because they abstract away the
>hardware, but they impose an extra buffering layer in some form or
>another.  That's why there's so much research on user-level network
>interfaces.
>
>I'm not saying that DOS is better than Linux.  I'm merely pointing
>out some problems associated with using UNIX variants for
>time-critical applications.  There are times when you want complete
>control over the hardware (or parts of it), and UNIX gets in the
>way.

I agree, there *are* some times when you want complete control over the
hardware.  But I think those circumstances are very rare indeed.  I
think games don't usually need that kind of access.



------------------------------

From: tseng@lysator.liu.se (\rjan Derel|v)
Subject: Is there a list of all Linux devicedrivers???
Date: 11 Jun 1994 16:04:55 GMT

I'm looking for a cd-rom driver for the Philips CM225MS/CM205MS
and I have not found any good source of information. Yes I know
there is the 'Hardware-HowTo' but it's very old and not very good.
The one I have is dated August 01, 1993.

------------------------------

From: jamesl@netcom.com (James Logajan)
Subject: Re: My problem? GCC problem? Linux problem?
Date: Fri, 10 Jun 1994 21:35:40 GMT

Rob Janssen (rob@pe1chl.ampr.org) wrote:
: In <ablumer.771128485@hubcap> ablumer@hubcap.clemson.edu (Aric D. Blumer) writes:

: >Given this code:

: >> void example(s)
: >> char *s;
: >> {
: >>    *s = 'h';
: >>    }
: >> void main()
: >> {
: >>    example("Hello!");
: >>    }

: If it is not too much work, modify your source.  Things like done in
: your example are ugly, IMHO.  That it works on a Sun is no excuse...

For what it's worth, this code also runs successfully on SCO ODT 3.2.
Did a quick scan on the cc man pages; doesn't seem to be any way of
telling it to put string constants in read-only memory. Hmmm...


------------------------------

Crossposted-To: comp.benchmarks,comp.sys.sun.admin
From: idletime@netcom.com (Totally Lost)
Subject: Re: Filesystem semantics protecting meta data ... and users data
Date: Fri, 10 Jun 1994 21:42:26 GMT

After a wild strike three Burkhard Neidecker-Lutz is OUT!

To recap this exciting drama ...

In article <1994Jun6.073403.4235@wavehh.hanse.de> Martin Cracauer writes:

Martin: SunOS-4.1.3 has a policy to have asynchronous writes for data, but
Martin: inodes and other superinformation is always written immedeatly. So, a
Martin: crash could only affect files that a written at that moment. I think
Martin: Solairs 2 does this, too. Does anybody know?
Martin: 
Martin: There was a patch for SunOS 4.1.3 to make the BSD-Filesystem writing
Martin: inodes async, too. That speeds up writing a large number of little
Martin: files by a factor of 2 to 3. Of course, a crash could really hurt now
Martin: that superinformation could be damaged.



Attempting to liven up the boring game and settle a few past scores,
John steps to the plate with <idletimeCqzHAL.I59@netcom.com>:

John: Both of these policies are stupidly wrong from a security stand point,
John: which is a significant case requiring orderly filesystem updates.
John: Actually both are wrong form ANY stand point, except to stupid sysadmins
John: and shit head systems programmers that don't give a damn about users data.
 ...
John: I've raised this point a dozen times, even at the usenix meeting where the
John: Berkeley guys first presented their concept of sync written meta data ...
John: and they didn't have a clue since they were one-tracked on making the
John: system clean from fsck point of view ... not from the users or data base
John: managers point of view. If the users/production data is corrupt, the
John: filesystem is corrupt!! PERIOD!!
 ...
John: I'd much rather have a totally unsafe filesystem that trashed on crash
John: and required a known good backup recovery over one that is guarenteed
John: to do the wrong thing and have dumb sysadmins think that just because fsck
John: ran fine all users/production data was fine too.



With <2t19urINNhr9@usenet.pa.dec.com> Burkhard takes the bait,
with a wild first strike:

Burkhard: Get a clue. Whoever writes sensitive data through the normal file
Burkhard: system with normal semantics deserves to get corrupted data. If you
Burkhard: want to commit data into a UNIX file system, you can do that perfectly
Burkhard: well by either using fsync(2) or by opening the file with O_SYNC
Burkhard: in the first place.
Burkhard: 
Burkhard: The asynchronous write batching of the normal UNIX file system is 
Burkhard: a very reasonable default behaviour and you can get as much consistency
Burkhard: as you like by using the appropriate controls.
Burkhard: 
Burkhard: Doing all writes synchronously has surprisingly bad performance (you
Burkhard: can try that for yourself on systems that support mounting whole
Burkhard: filesystems synchronously, such as Ultrix or DEC OSF/1.

[The excitement builds, he tells the UNIX croud they deserve corrupted data
setting the stage to introduce some vapor ware he has stock in ... ]



John steps backup to the plate with <idletimeCqzHAL.I59@netcom.com>:

John: My primary point which you failed to address was facility level reliablity
John: problems caused by normal operational proceedures recovering from
John: UNIX crashes ... due to poor filesystem designs, which include BSD/SUN/DEC.
John: After being primarily responsible for production UNIX systems from 1975
John: todate ... I have a long list of war stories for such (aka experiences).
John: 
John: Read first, then write .... one point of the posting was that ordered
John: writes correct 95% of the reason programmers are forced to use O_SYNC
John: to protect database updates. O_SYNC was a stupid solution to the problem
John: in the first place. Ordered writes with commit operations solves the
John: entire problem without the huge performance penalty.
John: 
John: To say "you can get as much consistency as you like by using the
John: appropriate controls" is a gross miss-representation since 95% of
John: the shipped systems do not have the option of mounting filesystems
John: synchronously and most of the remaining 5% that can, can not due
John: to the penalty.
John: 
John: Why is it that you can not understand the gains possible with ordered
John: writes? I guess it's just because you don't take the time to think ...
John: or can not ...


With <2t3qreINNql6@usenet.pa.dec.com> Burkhard takes the bait again
with a second wild strike:

Burkhard: My point was that most file system operations on UNIX system don't
Burkhard: require that much consistency and that for these the normal file
Burkhard: system semantics are ok as a default.
Burkhard:
Burkhard: Err, I do, and Advfs uses an even more clever mechanism without the
Burkhard: need of changing applications or the file system semantics (thank you,
Burkhard: Digital does understand production systems). The VMS operating system
Burkhard: has all sorts of facilities like that in the file system, but more
Burkhard: often than not the performance impact of having these semantics as
Burkhard: the *default* rather than available only where needed is very high.

[So he backs off that UNIX users deserve corrupted data, to they don't really
need correct data, and tring to save the day makes the standard side pitch that
for anyboding wanting correct data VMS will solve all UNIX users ills ...
with "thank you, Digital does understand production systems". I guess this
means Ultrix (DEC's unix offering) is not a production system ;-)  ]

BTW: for those who have not experienced VMS, try porting a large unix
application to it for a few years, I didn't like it (most of 1988 & 1989)
and the best VMS could do was 1/4 the performance of the same application on
UNIX with only 75% the usability. Coupled with the DEC memory and processor
hungry office automation product the result was a total dog, but the State
of Calif got what they ordered - with DEC helping specify it. The DOD/DOE/NSF
contract administrators came to the same conclution 10 years earlier when
they awarded the contracts for Berkeley to make BSD 4.x, over the DEC 10 and
Vax camp that tried to put a UNIX emulator on top of VMS at SRI.




John steps backup to the plate with <idletimeCr2vIs.5zJ@netcom.com>:

John: And my point is that any production (either interactive software
John: development users, student instructional servers, or more traditional
John: end-user applications) can not accept the security impacts or data
John: corruption associated with the current filesytedesigns. Are you tring
John: to say that having files undetectably corrupted with uninitialized
John: deleted file contents is an acceptable result of a "normal" operational
John: failure?
 ...
John: What I (and customers should) really demand is operational saftey and
John: security. Transactional security and semantics are only a plus, and not
John: addressed in my original post. And while I believe they should be part
John: of all UNIX offerings, I do not require it.
John:
John: What is the point you are trying to make? That the security and file
John: corruption aspects of current filesystem design during OS/Hardware
John: failure is acceptable?
John:
John: My only point is that this is not acceptable or necessary.




With <2t6lmqINNe3m@usenet.pa.dec.com> Burkhard takes the bait again
with a third wild strike:

Burkhard: No modern BSD file system will show up with deleted file contents in
Burkhard: a file after any kind of disk crash,
 ...
Burkhard: There shouldn't be any security breaches (and to the best of my
Burkhard: knowledge any recent UFS implementation doesn't have any). To
Burkhard: solve file corruption problems of the kind you mention (multiple
Burkhard: writers) requires a lot more than just writing some metadata in
Burkhard: any particular order (Berkely UFS is very careful to order the
Burkhard: writes it does to disk so that that can't happen).
Burkhard:
Burkhard: Also, you don't seem to be familiar with really *current* UNIX file
Burkhard: systems like Advfs or JFS which suffer from none of your problems.

[The wimp goes from saying that UNIX users deserve it, to they don't really
need correct data, to those that have modern BSD filesystems have correct
data after all - NOT!  And then goes on to push the DEC Ultrix solution again,
with a claim of correctness that can not be quickly verified since the sources
are not public - if anyone does happen to have source access could they verify
the claim?]



Mr Neidecker-Lutz really puts his ingorance of the issue right to the point
here. I have followed the BSD filesystem since it's beginnings and have never
seen a version that was data safe over a crash. Maybe he can point us to
the internet server that has the data safe UFS implementations?

The primary place I have seen the corruption over the years is in log files and
peoples mail box (while I was only made aware of 3 cases where a user got
sensitive data in his mail box or other file, I have lost track of a fairly
large number of files that had binary data inserted into them).

From DEC's own archive on gatekeeper.dec.com via ftp you may examine various
"modern" versions of USF:

        /.0/BSD/NetBSD/NetBSD-current/src/sys/ufs
        /.0/BSD/FreeBSD/FreeBSD-current/src/sys/ufs

Pay particular attention to the flow path of:

        ufs_vnops.c:ufs_write(vp, uio, ioflag, cred)
        do {
                lbn = lblkno(fs, uio->uio_offset);
                on = blkoff(fs, uio->uio_offset);
                n = MIN((unsigned)(fs->fs_bsize - on), uio->uio_resid);
                if (n < fs->fs_bsize)
                        flags |= B_CLRBUF;
                else
                        flags &= ~B_CLRBUF;
                if (error = balloc(ip, lbn, (int)(on + n), &bp, flags))
                        break;


as it updates the block numbers in the indirect blocks via the path:

        ufs_bmap.c:balloc(ip, bn, size, bpp, flags)

                ...

                /*
                 * Fetch through the indirect blocks, allocating as necessary.
                 */
                for (; ; j++) {
                error = bread(ip->i_devvp, fsbtodb(fs, nb),
                    (int)fs->fs_bsize, NOCRED, &bp);
                if (error) {
                        brelse(bp);
                        return (error);
                }
                bap = bp->b_un.b_daddr;
                sh /= NINDIR(fs);
                i = (bn / sh) % NINDIR(fs);
                nb = bap[i];
                if (j == NIADDR)
                        break;
                if (nb != 0) {
                        brelse(bp);
                        continue;
                }
                if (pref == 0)
                        pref = blkpref(ip, lbn, 0, (daddr_t *)0);
                if (error = alloc(ip, lbn, pref, (int)fs->fs_bsize, &newb)) {
                        brelse(bp);
                        return (error);
                }
                nb = newb;
                nbp = getblk(ip->i_devvp, fsbtodb(fs, nb), fs->fs_bsize);
                clrbuf(nbp);
                /*
                 * Write synchronously so that indirect blocks
                 * never point at garbage.
                 */
                if (error = bwrite(nbp)) {
                        blkfree(ip, nb, fs->fs_bsize);
                        brelse(bp);
                        return (error);
                }

The sync bwrite makes sure the indirect block is atleast zero if a crash
occured shortly after this point ... reading on ...

                bap[i] = nb;
                /*
                 * If required, write synchronously, otherwise use
                 * delayed write. If this is the first instance of
                 * the delayed write, reassociate the buffer with the
                 * file so it will be written if the file is sync'ed.
                 */
                if (flags & B_SYNC) {
                        bwrite(bp);
                } else if (bp->b_flags & B_DELWRI) {
                        bdwrite(bp);
                } else {
                        bdwrite(bp);
                        reassignbuf(bp, vp);
                }


note that flags & B_SYNC above will normally be false and the indirect block
pointing to our new data block will be written at the mercy of either the
disk sort routine or a combination of buffer cache flush followed by disk sort.

Back in ufs_write we return from balloc with the newly allocated bp:

                if (error = balloc(ip, lbn, (int)(on + n), &bp, flags))
                        break;
and proceed with

                bn = bp->b_blkno;
                if (uio->uio_offset + n > ip->i_size) {
                        ip->i_size = uio->uio_offset + n;
                        vnode_pager_setsize(vp, ip->i_size);
                }
                size = blksize(fs, ip, lbn);
                n = MIN(n, size - bp->b_resid);
                error = uiomove(bp->b_un.b_addr + on, n, uio);
                if (ioflag & IO_SYNC)
                        (void) bwrite(bp);
                else if (n + on == fs->fs_bsize) {
                        bawrite(bp);
                } else
                        bdwrite(bp);

copying the data into the newly allocated buffer and since ioflag & IO_SYNC
is generally false asyncronously write it (possibly delayed in the cache)
allowing the disk sort to write it in some random order from this inodes
point of view.

Should a crash occur after the above indirect block is written to disk, and
before this data block is written, the file WILL undectectably contain
the previous data on the disk. FSCK will be happy, the system admin will
have no means to identify this file as corrupt, and the user can get someone
elses deleted data - security breach first class.

This chance order is the best case, and as the original author pointed out:

Martin: SunOS-4.1.3 has a policy to have asynchronous writes for data, but
Martin: inodes and other superinformation is always written immedeatly. So, a
Martin: crash could only affect files that a written at that moment. I think
Martin: Solairs 2 does this, too. Does anybody know?

which insures the wrong order.

In addition significant performance delays are implictly created by
extra writing of indirect blocks that would not occur if all the
data blocks rapidly allocated to an indirect block were written prior
to allowing the indirect block to be written, and the same policy
repeated up the indirect chain.

Nearly 100% of all UNIX systems in the field have this problem.

Installing a UPS on your existing system will remove some small percentage
of unexpected crashes. On most busy server systems known OS bugs resulting
in a panic are by far the common problem, followed by random hardware
failures. At each of these types of failures you have no way of knowing
what data receintly created is corrupt. Fsck can only tell you the
filesystem meta data is correct ... it tells you nothing about the
contents of those same files.



Next batter step up to the plate please ...

John

------------------------------


** FOR YOUR REFERENCE **

The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:

    Internet: Linux-Development-Request@NEWS-DIGESTS.MIT.EDU

You can send mail to the entire list (and comp.os.linux.development) via:

    Internet: Linux-Development@NEWS-DIGESTS.MIT.EDU

Linux may be obtained via one of these FTP sites:
    nic.funet.fi				pub/OS/Linux
    tsx-11.mit.edu				pub/linux
    sunsite.unc.edu				pub/Linux

End of Linux-Development Digest
******************************
