Date: Wednesday, 21 Aug 1985 16:37:04-PDT
From: mitton%beorn.DEC@decwrl.ARPA
Subject: Interrupt Driver Async I/O


Everything You wanted to know about PC Async Comm, but were afraid to ask....
----------------------------------------------------------------------------

A few months back I sent in a note asking for help about how to do reliable
interrupt driven communications using the IBM PC async comm port.  Now I
know more than you may want to hear.  But I would like to share my knowledge
with the net to help out the adventurous, and expose some of the problems that
make it so difficult, so that these situations might not get designed into
to future products.  Scattered though out this message are some of my own
editorial comments in [square brackets].

- The UART used in the IBM PC family is the Western Digital 8250 and friends
(recently the National Semi 8250B, and the 14650 in the AT).  This UART is
single buffered.  There is only one receive character buffer to store a byte
in while the next byte is being assembled in the shift register.  This 
means that you have only ONE CHARACTER TIME (on average) to process that byte 
before the next one wipes it out (an overrun).

I bet you thought your AT adapter card or AST Advantage has an 8250B on it?
The IBM PC-AT Tech Reference manual doesn't say what the chip is (unlike the
XT) or that it is different until you look at the schematic.
Surprise!  Look again, its a NS 14650.  National describes the chip on the
same sheet as the 8250B.  It seems to be functionally identical, but with
a faster access time.  Unfortunately, it is still not fast enough for the
'286.  Read IBM's ISV notes on avoiding doing back-to-back I/O references
to the same chip so as to not violate chip access speeds.

	[ Modern UARTs should have at least a 4 character FIFO.
	This makes for less interrupt latency problems and more reliability]
	[Also better baud rate generators and dividers are available 
	these days.  19.2k should be easily and accurately supported.]

- Next I did the following rough instruction budget.  An IBM PC-XT is an
8088 running at 4.77Mhz,
			 1 clock cycle =  210ns
   1 memory reference = 4 clock cycles =  840ns
1 average instruction = 4 memory refs  = 3360ns 

1 second / ( 3360ns / avg ins)	~= 297,619 avg ins/sec

 9600 bps = 960 cps

(297,619 avg ins/sec) / 960 cps	 =  310 avg ins / char

Now that's only for half duplex!  Halve that for full duplex load.
And don't forgot to subtract for the time PC memory refresh uses.
It becomes rather clear that the interrupt service code path must be as short
as possible.  At least less than 150 instructions average.

Now you may want to quibble with my assumption of 4 clocks/average instruction.
Truth be known, I made it up.  Everything I see indicates that on an 8088
the average is more than that, and knowing that makes this estimate seem
optimistic.  

On the AT, things get much better because the 16 bit bus causes more to happen
in fewer cycles.

[ Now do understand why a FIFO is needed?]
[ My code on a Rainbow (with a 3 char FIFO UART) code was written in C 
(including interrupt service) and worked fine after first debugged it]

- Now if this isn't bad enough, lets toss in a interrupt handling problem:
The IBM PC 8259 Interrupt Controller is programmed in the BIOS to be edge-
sensitive.  The 8250 seems to supply edges properly, except when there may
be multiple interrupts to service (ie: a full duplex receive complete and
transmit complete at the same time).  Now this behavior is not documented
on any spec sheet I've seen, (usually because they don't tell you what happens
in this case)  but rumor has it that National changed the 8250B
(and the 14650) so that it does not toggle the interrupt line when presenting 
such stacked interrupts.  It is unknown whether the WD 8250 does the same.
But the existence of said crock, has been experimentally verified many times.
It is discussed thoroughly but sadly inconclusively in the file EDGES.INT.

[ could someone get the UART people to fess up in writing?]
[ also, tell them not to do it again!]

- Okay, so what do you do?  Well, you have to write your interrupt service
routines as a loop, servicing the UART until all pending interrupts have
been handled and you won't lose an interrupt edge.  The COMPKG2 and the 
MIT PC/IP service routines do a good job, but they have some flaws.

  1) They recheck the UART status by reading the LSR at the bottom of the loop.
Since reading the LSR register will reset any pending receive error conditions,
you could easily lose notification of an overrun, framing, or parity error.
It is much better to re-read the ISR instead, because it serializes the 
the highest priority current status.

  2) You should EOI the 8259 interrupt controller at the beginning or in
the middle of the service loop.  Notice that the 8250 clears the interrupt 
condition upon servicing (reading or writing) the appropriate register.
If you EOI afterwards, then there is a window in which an interrupt may
arise from the UART, but get dismissed when you clear the interrupt controller.

  Another bug to avoid, which I made once myself, is: do not break the loop 
in the character processing.  The routine will hang with a unserviced interrupt
pending on the UART and no more edges to trigger the 8259.  (unless you 
implement the timer described below)

The proper loop as I coded it is as follows:

	send EOI to 8259
loop:
	read IIR
	switch(IIR)
	{
		case NO_INTERRUPT:
			iret;
		case XMIT_READY:
			send next char;
			break;
		case RECV_READY:
			receive and buffer char;
			break;
		case LINE_STATUS;
			record error condition;
			break;
		case MODEM_STATUS:
			record state change;
			break;
	}
	goto loop
			
Note:
	- EOI done outside the loop may generate extra nop interrupts,
if no stacked interrupt, but one arrives during a long service path.  
An inside the loop EOI eliminates this but adds more code to the loop.

	
- Great, so-far-so-good.  What could screw us up?  Well, I forgot to mention
that what I was writing was a device driver and it runs in the background
on the async ports.  Because we are not dealing with a big system with
device allocation concepts, there are all sorts of DOS programs and 
utilities that can stomp on your comm port.  The MODE command will
do you in, especially if you forget to take the command out of your AUTOEXEC
[took me a month to figure that one out].  BASICA grabs the comm port.   
Even Symphony thinks that it can grab the comm port for it's terminal emulator,
unless you do their not-well documented re-configuration procedure.

[software writers: please don't assume that the UART is available!]

- Another program that caused us to lose, was PROKEY.  It had hooked on to
Interrupt 1C, the user clock tick handler, which we were using too, and spent 
soooo much time on it, that our timing just totally screwed up.  This was 
finally solved when we fixed another problem below.

[Be careful of doing to much on the clock tick.  It could screw up someone
else.]
	
- One thing that I did to add some robustness to all this (and find some bugs)
was to add a timer scheme.  Essentially, whenever I started a transmission
or reception of a message, I initialized a word to a nonzero timeout value.
A clock tick routine decremented the cell, if nonzero, and if it went to
zero, reset it and faked an interrupt to the service routine.  This feature
allowed the code to recover (as opposed to hanging forever) from lost 
interrupts and errant MODE commands.  Hopefully, this should never happen.
But it did at first, and a trace of the current state helped.

[real disasters give you first hand experience on how to defensively program.]

- Now the device service looks good, but I still am getting overruns at 7200
and 9600.  So I thought some more about where the time goes in the CPU.
Another way that you lose CPU instructions is to other interrupt service
routines and code sections that disable interrupts.  Unfortunately,
unlike a VAX, we don't have the multiple IPL levels to synchronize CPU
threads without shutting out device service.  

   In my driver there are 3 levels of synchronization that use interrupt
locking around cross-level queue operations.  Unfortunately, the interrupt
locking queue function was the default even for queue operations in the 
same level.  A better analysis of interrupt locking and necessary 
synchronization lead to fewer and shorter interrupt disabled code sections
and a much better performance level.  I have now even figured out a better
semaphore interlock with the interrupt service routine that will eliminate
even more interrupt disabled code.  

	[Interrupt latency on the 8086 architecture is precious!
	 You must try to minimize all interrupt disabled code paths.]
	[Another reason to have a FIFO in the UART!]

- Finally, I had done almost all I could think of; I had tweaked the
interrupt service loop, bummed the code paths to a minimum, and was still
getting overruns on the XT.  I still had this feeling that they might be
systematic, so I put a little code in the overrun routine that recorded
the segment and offset of the code interrupted just before the overrun
was serviced.  A higher level monitor printed it out.  I was perplexed
because it was always the same: FE00:FEEA   I had sort of expected to
find some code that had just done a STI, but instead I was staring at the
stack cleanup code for the clock tick service in the ROM BIOS. (below)

FEA5	TIMER_INT PROC FAR
FEA5	FB	STI		;Interrupts back on
.... push regs, increment DOS time in RAM, turn off floppies...
FEE3	CD1C	INT 1CH		;Transfer control to a user routine
FEE5	B020	MOV AL,EOI
FEE7	E620	OUT 020H,AL	;End of interrupt to 8259
FEE9	5A	POP DX
FEEA	58	POP AX
FEEB	1F	POP DS		;Restore machine state
FEEC	CF	IRET

   Walking back up the code, I don't see anything unusual.  Wait a minute!
Why are we finally EOI'ing the 8259 after the INT 1C, when we STI'ed back
at entry?  Oh, that's to make sure that we don't reenter huh?  Well what
about the 8259 all the time that the INT 1C handlers were running?

   Yes folks, The Single Serializing Priority Interrupt Controller has been
blocked the entire time.  This was preventing any other device interrupt
service during the clock tick handling.  Initially, the 1C handler is
an IRET, and it's okay.  But in practice, my driver and other things were 
on there with a substantial total code path, almost guaranteeing lossage.

   My fix to this BIOS crock, had to replace the entire interrupt 08 routine,
since it dispatches the interrupt 1C and has that EOI in the end.
I wrote the following code in MWC assembler that handled problem of reentrancy 
as well:

/ Interrupt 08 Handler
/ This routine *REPLACES* the IBM PC BIOS interrupt handler for the
/ clock frequency interrupt.  It *MUST* be loaded into the system before
/ any other Int 08h user.
/ We must replace the BIOS routine because it has a nasty bug in that
/ it does not reset the 8259 Interrupt Controller until the Int 1Ch Handler(s)
/ are done.  This effectively locks interrupts for the entire period.
/ Chaining to the original handler would not work.
/
TIMER_LOW	= 0x006C
TIMER_HIGH	= 0x006E
TIMER_OFL	= 0x0070
MOTOR_STATUS	= 0x003F
MOTOR_COUNT	= 0x0040

	.shri
ticks:	.word	0		/ ticks flag

	.globl	cex_clock_
cex_clock_:			/ Timer interrupt entry point
	push	ds
	push	ax
	push	dx
	mov	ax, $0x40	/ set BIOS DATA segment value
	mov	ds, ax

	inc	TIMER_LOW	/ Increment BIOS Time of day
	jnz	T4
	inc	TIMER_HIGH
T4:
	cmp	TIMER_HIGH, $0x18
	jnz	T5
	cmp	TIMER_LOW, $0xB0
	jnz	T5

	sub	ax, ax
	mov	TIMER_HIGH, ax
	mov	TIMER_LOW, ax
	movb	TIMER_OFL, $1
T5:				/ Test for diskette timeout
	decb	MOTOR_COUNT
	jnz	T6
	andb	MOTOR_STATUS, $0xF0
	movb	al, $0x0C
	mov	dx, $0x03F2
	outb	dx, al
T6:				/New code starts here----
	movb	al, $0x20		/ reset 8259 Interrupt Controller
	outb	0x20, al		/ this allows other interrupts to queue
	inc	cs:ticks		/ increment tick semphore
	cmp	cs:ticks, $1		/is it the first time?
	jne	T8			/No, just leave

T7:	pushf				/fake interrupt call to local routine
	cli				/
	push	cs			/
	call	cex_clicker		/ giving us first priority

	int	0x1c			/ call everyone else

	cli				/ disable interrupts, if reenabled
	dec	cs:ticks		/ check if others came in
	jnz	T7			/ yes, do it again

T8:	pop	dx			/ restore state
	pop	ax
	pop	ds
	iret				/ exit interrupt service


This routine also solved the PROKEY problem mentioned above, since
I hooked my driver directly in the clock tick chain before anyone else.
The 4 instructions after T7 can be removed, and this can be used as a
general purpose Int 08 handler replacement.   After installing this,
I noticed that my polygon terminal emulator no longer gets overruns either.

There may be some 1C clock tick users that would be upset by the change,
but I haven't found one yet.  (Even the PC Network tolerates it)
Another way to work around this problem, if you don't need priority over
the 1C handlers, is to hook Int 08, call it and wait for it to return.
There are probably yet another way around this problem, but this seems the 
cleanest to me.

	[DOS needs better clock tick services!]
	[and don't hog them!]

Dave Mitton, DECnet-DOS Development.
Enet: OLORIN::MITTON 	
Arpa: mitton%olorin.DEC@decwrl.ARPA
Usenet: decwrl!dec-rhea!dec-olorin!mitton
/opinions expressed here are mine, although DEC might feel likewise/

Posted:	Wed 21-Aug-1985 19:33 Eastern Standard Time, Tewksbury, Mass.
To:	RHEA::DECWRL::"info-ibmpc@usc-isib.arpa"
