From: Digestifier To: Linux-Development@senator-bedfellow.mit.edu Reply-To: Linux-Development@senator-bedfellow.mit.edu Date: Thu, 28 Oct 93 18:13:59 EDT Subject: Linux-Development Digest #196 Linux-Development Digest #196, Volume #1 Thu, 28 Oct 93 18:13:59 EDT Contents: TERM problems- Please help! (Christopher Lau) Re: Slowness in scsi disk access (Olaf Frommann) Help for VLB IDE-Controller (UMB) needed! ( Karsten Ballueder) [Q] I need an inet guru to install PVM between 2 linux box (Buffat Marc) Re: Slowness in scsi disk access (Piercarlo Grandi) Yet another core dumps name suggestion (Basile STARYNKEVITCH) Re: Slowness in scsi disk access ("Brian E. Gallew") Scsi performance doubled! (Was: Slowness in scsi disk access). (Eric Youngdale) ---------------------------------------------------------------------------- Crossposted-To: comp.os.linux.help,comp.os.linux,comp.os.linux.misc From: clau@acs.ucalgary.ca (Christopher Lau) Subject: TERM problems- Please help! Date: Thu, 28 Oct 1993 04:04:24 GMT Anyone have any clues on this one: Here's the details: o On Monday, term was working fine o On Monday night, shutdown the system properly, powered down for the night. o On Tuesday morning, term doesn't work. It just sits there and hangs, nothing appears to go over the serial line (no flashing lights on the modem) o On Wednesday, term still doesn't work.. What I did: o Something hung? Check /usr/spool/uucp for lock files, ps -ax for other processes.. nothing wrong.. Reboot. Same problem. o Perhaps my autoterm script isn't working anymore.. try it manually: - dial from kermit, term on remote, ^Z suspend, term /dev/cua1 Nothing.. same problem - Try an experiment: On remote 'sz bigfile', ^Z suspend, on local 'rz /dev/cua1'. Works fine.. Try it with sending files- also works, so it's not the I/O system o Perhaps network configuration got changed, new escape sequences needed- - modify termrc's to escape all control chars (0-31 and 128-159) Still doesn't work. - use my own line checking program- everything is fine no new escapes - change everything back o Try a completely different remote host with different architecture, in different building, on different subnet, different everything- Still doesn't work. o Perhaps file got corrupted, restored backup from tape- same checksum, same size, when you replaced the current version with the backup, same thing, term just freezes. o Perhaps libraries changed- recompile. New file has same checksum, same size same problem. o Repeat the above two on Linux box.. no go in either case. o Maybe my Linux libraries got scrambled somehow - Restore libc.so.4, libc.so.4.X.X from install disks - restore libc.a, libc.sa from install disks - Nothing.. nada.. zip! o Check all permissions in /dev/- everything fine. o Check for modified files in directory structure. There are some (utmp, wtmp, .history, things like that) but none that would cause term to fail completely. o Maybe my hardware died (don't know how it would do this and still let me log in normally using kermit, etc, but who knows) - Replace UART chip. Still nothing - Use other port on same board. Still nothing - Replace with a brand new board. Still nothing. - Try modem with no error correction, at 2400 baud. Nothing. - Try different 2400 baud only modem- still nothing. - Turn off turbo button (bus speed now 8MHz). Still nothing - Disable cache. Still nothing. I'm out of options and I'm seriously thinking of doing a complete re-install. Does anyone else have any ideas about what's wrong before I try re-installing? You can assume that I remembered to change the termrc file on both sides and did proper shutdowns and reboots for any procedures above that need these, if I've neglected to put in the fact. I'm convinced that my machine is somehow at fault, since I tried it with two different remote hosts, with different OS and both times, term failed. If I turn on term's packet tracing, I see messages to the effect of "put XXX (total in queue) in sendQ", and each time the total increases, it never goes down. It appears that data never gets sent out. But why? I've recompiled, that doesn't help, everything else on the system seems to work fine. The only thing I can think of is to turn back the clock to Monday's date and see if things work again.. (if it does, I'll have to take my Linux disks to the shredder and a hammer to my computer) I'm getting pretty desparate here.. I need term so I can view my correlation graphs and my maple and matlab plots over the modem, so I can get news, etc.. Any and all help would be greatly appreciated! Thanks c4 -- Christopher Lau | Dammit Jim, I'm a doctor, The University of Calgary | not an engineer! Dept. of Electrical & Computer Engg. | Well, you're an engineer now.. lau@enel.ucalgary.ca -OR- clau@acs.ucalgary.ca -OR- root@fusion.cuc.ab.ca ------------------------------ From: sm1of@tuhhco.rz.tu-harburg.de (Olaf Frommann) Subject: Re: Slowness in scsi disk access Date: 28 Oct 93 08:26:04 GMT In article , pcg@aber.ac.uk (Piercarlo Grandi) writes: | >>> On 27 Oct 1993 03:15:34 GMT, oreillym@tartarus.uwa.edu.au (Michael | >>> O'Reilly) said: | [very interesting stuff deleted] | | Later on you have statistics that state that only in 15% of cases a | request for a buffer is satisfied by looking at the first one in the | inaptly named 'free list'. | | WHAT? It has to *scan* the free list for a buffer? Cannot it just take | the first buffer and run scared? WHY? :-) :-) | | Searching the free list? Just Say No. | | Michael> A much-vaunted 'free list' doesn't help if it's empty. | | But at least one does not have to scan 15,000 entries (which costs 50ms) | on the 'free list' just to discover it is empty. | | Also, scanning for a clean buffer is very, very bad -- this victimizes | clean buffers in preference to dirty ones, and is a no-no. Ideally the I absolutely agree. Maybe this is stupid, but what about the TWO lists idea? Lets have a free list, where we can take buffers if its not empty and a used buffer list, which is scanned for old dirty buffers from time to time, to fill the free list? Just my $.02 Olaf -- Olaf Frommann, PHONE: +49 40 7718-2942 TU Hamburg-Harburg, FAX : +49 40 7718-2573 Arbeitsbereich Stroemungsmechanik I e-mail: Frommann@tu-harburg.d400.de ------------------------------ From: karsten@lt2.cs.rhbnc.ac.uk ( Karsten Ballueder) Subject: Help for VLB IDE-Controller (UMB) needed! Date: 28 Oct 1993 10:55:49 GMT Reply-To: kballued@charon.physik.uni-osnabrueck.de ( Karsten Ballueder) Hi, I have a VESA LocalBus IDE-Controller with an UMB Chipset, which uses DOS-driver to set up the appropiate parameters for the harddisk. It works fine by loading the driver and then using bootlin to boot linux, because the driver only sets up some registers (so it's no REAL driver). As I would like to write a driver/setup utility for LinuX (I did it some time ago for the ADI2 chipset.), I need some information about this card/its chipset. If somebody can help me, please contact me via email to the addresses below. If you are good in disassembling DOS-drivers (I never tried) and would like to help me (the driver is pretty short and should be easy to understand.), PLEASE contact me. Thanks in advance,Karsten -- _____..---========+^+=========---.._____ ______________________ __,-='=====____ ================== _____=====`= (._____________________I__) - _-=_/ `--------=+=--------' / /__...---===='---+---_' '------'---.___ - _ = _.-' `--------' USS Enterprise, NCC-1701D ------------------------------------------------------------------------------ Karsten Ball"uder, Royal Holloway, c/o Physics Department, Egham Hill, Egham, Surrey TW20 0EX, United Kingdom charon.physik.uni-osnabrueck.de jupiter.rz.uni-osnabrueck.de karsten@dcs.rhbnc.ac.uk --> LinuX - The better text adventure. <-- ------------------------------ From: buffat@europe.mecaflu.ec-lyon.fr (Buffat Marc) Subject: [Q] I need an inet guru to install PVM between 2 linux box Date: 28 Oct 1993 11:47:25 GMT I try to install pvm (parallel virtual machine) between 2 linux box running linux pl13j and connected via ethernet. The net software (ping, rlogin, rsh ..) works correctly between the 2 machines. PVM works well on one machine using the localhost, but I cannot use it with the 2 machines. Here is the problem: To connected the 2 machine, PVM uses socket and creates a socket with: sockid=socket(AF_INET,SOCK_DRGRAM,0); and try to link the socket with the interface name eth0 using sockaddr sad; strcpy(sad.sa_data,"eth0"); sad.sa_family=AF_INET; bind(sockid,&sad,sizeof(sokaddr)); I got an error from bind : "unable to assigned the request name" If I change the intergace name from eth0 to lo (local loopback), it works Any idea or suggestion will be appreciated ? ============================================================================= Marc BUFFAT ++++++++++++++++++++++++ Lab. Mecanique des fluides LMFA | CNRS URA 263 | ECL, 36 av. Guy de Collongue | ECL Lyon | Ecully 69131, FRANCE | UCB Lyon I | tel: (33) 78/33/81/27 ++++++++++++++++++++++++ fax: (33) 78/64/71/45 email: buffat@mecaflu.ec-lyon.fr ------------------------------ From: pcg@aber.ac.uk (Piercarlo Grandi) Subject: Re: Slowness in scsi disk access Reply-To: pcg@aber.ac.uk (Piercarlo Grandi) Date: Thu, 28 Oct 1993 12:41:10 GMT >>> On Thu, 28 Oct 1993 01:44:25 GMT, bsa@kf8nh.wariat.org (Brandon >>> S. Allbery) said: Brandon> In article pcg@aber.ac.uk Brandon> (Piercarlo Grandi) writes: pcg> WHAT? It has to *scan* the free list for a buffer? Cannot it just pcg> take the first buffer and run scared? WHY? :-) :-) Brandon> Because it might be a page from an executable, or an mmap()ed Brandon> file. Ok, it seems as if the smileys did not get it across: the problem is that the free list is overloaded with several distinct meanings; it is a free list only incidentally; this is IMNHO a misfeature. Ideally one would manage all buffers/pages as a single resource, using mmap thruout, and advising/PFF/WSCLOCK whatever for victimization... Eventually this will happen hopefully -- it would make for a nice, and not too difficult, M.Sc. project. Any takers? Brandon> Neither of which is LRU-updated, since the cost is prohibitive Brandon> (*every* time the page is touched it would have to be put on Brandon> the end of the LRU-list). But there are approximations to LRU (and to WS, which is a rather popular approximation to LRU) that are rather cheap; my favourite is PFF; WSCLOCK also looks good. Also, LRU is as a rule rather inappropriate for file access, which is tipically FIFO and, far less often, RANDOM. I would suspect that the best thing would be to associate a policy (one of PFF/FIFO/RANDOM) with a file or segment, either by way of cleverly chosen defaults (e.g. PFF for executables and segments, FIFO if file is opened in O_WRONLY or O_RDONLY mode, RANDOM if the file is opened O_RDWR), overridable by advising (lisp systems tipically would want to advise RANDOM on data segments just before a GC). The memory pool could then be managed with an easy&quick version of the buddy system, one that for example does not coalesce buddies if they are larger than a threshold (say 16KB), as there is no point in having large contiguous blocks, and coalescing/splitting costs time. Then the free list(s) would be really free lists(s), and so on. Heaven on earth, and free beer for everybody? :-) ------------------------------ From: basile@soleil.serma.cea.fr (Basile STARYNKEVITCH) Subject: Yet another core dumps name suggestion Date: Thu, 28 Oct 1993 12:44:10 GMT Reply-To: basile@soleil.serma.cea.fr Perhaps the core dump file name format (in the sense of kernel sprintf format) could be made as a configurable resource. It might be configurable in a system wide manner (perhaps by sysconf call) or on a process by process basis (then the core format would be yet another process data, either setable by yet another system call, or a special device, or better thru the /proc file system; eg the core format of process xxx could be in /proc/xxx/coreformat). Unfortunately i don't have time yet to implement these tricks. In the mean time, i prefer to have some unusual punctuation characters (like %#@) in core file names, eg if program bogus made a core in process 345 the core file could be core%bogus#345. Also, my opinion is that the whole idea of core files (a good idea in the PDP-8 unix days) is wrong today, since more and more programs are huge and buggy (todays core file can easily much bigger than the executable file whose execution produced them). Perhaps each process could have an associated debugger server (or connection) which would be sent a message by the kernel when a core used to be made (ie when a naughty signal is sent). A default implementation of such a debugger server might produce core files just like gcore command. Better choices would be forking an interactive debugger (eg an xxgdb). --- Basile STARYNKEVITCH ---- Commissariat a l Energie Atomique DRN/DMT/SERMA * C.E. Saclay bat.470 * 91191 GIF/YVETTE CEDEX * France fax: (33) 1- 69.08.23.81; phone: (33) 1- 69.08.40.66 email: basile@soleil.serma.cea.fr; homephone: (33) 1- 46.65.45.53 N.B. Any opinions expressed here are solely mine, and not of my organization. N.B. Les opinions exprimees ici me sont personnelles et n engagent pas le CEA. ------------------------------ From: "Brian E. Gallew" Subject: Re: Slowness in scsi disk access Date: Thu, 28 Oct 1993 08:59:43 -0400 I personally like the idea of the bdflush-type process. This leads to a question: What does the idle task do? I seem to recall that at one time Linus was going to actually have it do something useful in any spare system time. Perhaps this could be made helpful here? -Brian ========================================================================= | "Are they dead?" | | "Does it matter?" | | - Pugsley and Wednesday in "The Addams Family." | ========================================================================= ------------------------------ From: eric@tantalus.nrl.navy.mil (Eric Youngdale) Subject: Scsi performance doubled! (Was: Slowness in scsi disk access). Date: Thu, 28 Oct 1993 14:52:09 GMT In article <1993Oct28.013903.11930@kf8nh.wariat.org> bsa@kf8nh.wariat.org (Brandon S. Allbery) writes: >>I think this is what SVR3 called the "buffer dribbler" or bdflush. It is > >Eric and I are working on this. At least I think he is still working on it... >no mail since this morning but he may well be as busy as I am (aargh!), or be >fixing some of the other things he recently found. I got slightly sidetracked rewriting some stuff for scsi. I will get into details later, but with my latest patches, it really burns rubber :-). In the past I had basically assumed that each buffer would be in memory in a nondeterministic position with respect to any other buffer. This means that if a scsi controller can do 16 segments of scatter gather, we are limited to 16Kb transfers. If you assume this, then only way to improve the I/O rate is to increase the block size. Piercarlo essentially suggested that instead of increasing the block size we cluster the buffers so that the buffers for adjacent blocks on the disk are also adjacent in memory. The scsi code could take advantage of this, and treat these sets of smaller blocks as one large block, and thus we could get larger I/O transfers while still using a smaller disk allocation block. Modifying the scsi code to take advantage of this is one thing, but the question is how we ensure that the buffers for adjacent blocks are adjacent in memory. It turns out that grow_buffers creates four buffers out of one page of memory and sticks them on the free list (in the opposite order, but this was easy to fix), so in a sense the system has a slight preference for generating clustered buffers. Also, there is code in the buffer cache for sharing code pages with the buffer cache which request four buffers in one page of memory - I am not sure, but I suspect that this may lead to patterns of sets of 4 buffers on the free list that are all from the same page. Anyway, I went ahead and modified the scsi code. It was trickier than I first thought because of the blecherous 16Mb limit and the DMA bounce buffers, but I got this all straightened out and it works on my 20Mb system. As I mentioned before, I made a very slight patch to grow_buffers so that the buffers for the new page are inserted onto the free list in the correct order (a better solution would be for the file_read routine to explicitly ask for a page worth of buffers, but this can wait until later). Anyway, I have been testing with iozone using the following command: ./iozone 25 1024 /dev/sda2 and prior to the patches I would tend to get something in the 500Kb/sec range (note that there are problems with the file/block device writing routines that severely limit performance if you are writing something less than 1024 bytes). Also, I have a 5Mb ramdisk, so the buffer cache never gets larger than about 15Mb. After the patches I get something like 1.2Mb/sec writing and 1.05Mb/sec reading. Note that there is no guarantee that the buffers are clustered in memory, and the block_read function does not even ask for it actually - it just happens to work out that this is what you get (at least to some degree). I have no idea what the actual statistics for the read/write are in terms of the actual transfer sizes - this is one thing I would still like to do. I believe that the disk itself should max out at something like 1.8Mb/sec with 64Kb transfers, and I am wondering why we are not getting it - incomplete clustering would certainly explain it. Anyway, I have uploaded the patches (you had to read all of the above to get to this :-) to tsx-11 in pub/linux/ALPHA/scsi/cluster.diff. They are reasonably debugged since I was running the relevant code that went into sd.c in a user process for a while so that I could test all of the various cases and make sure that it handled them correctly. You should probably use something like pl13 or even later with this. If the patches do not apply cleanly, beware - I made them very late last night, and I might have missed something. Before these go into the distribution kernel, we probably want to fix the various filesystems to actually request clustered buffers - these modifications should be quite minor. I doubt that I will have to twist any arms to get people to try this :-). Note: only the disk code was modified - in theory the cdrom code could also be modified but the base I/O rates are much lower so you would not gain as much. -Eric -- "The woods are lovely, dark and deep. But I have promises to keep, And lines to code before I sleep, And lines to code before I sleep." ------------------------------ ** FOR YOUR REFERENCE ** The service address, to which questions about the list itself and requests to be added to or deleted from it should be directed, is: Internet: Linux-Development-Request@NEWS-DIGESTS.MIT.EDU You can send mail to the entire list (and comp.os.linux.development) via: Internet: Linux-Development@NEWS-DIGESTS.MIT.EDU Linux may be obtained via one of these FTP sites: nic.funet.fi pub/OS/Linux tsx-11.mit.edu pub/linux sunsite.unc.edu pub/Linux End of Linux-Development Digest ******************************