PCI Bridge Technology White Paper
Jim Murashige, Adaptec


PCI Architecture Overview

   A fundamental element of the PCI architecture is the concept of Bridges.
Very simply, Bridges are system building blocks that allow traffic on one
bus to cross over to another bus.  The buses that a Bridge connects together
may be very similar or very different.  This feature allows a system to have
any combination of CPU, Legacy I/O and PCI buses.  Bridges handle all of the
protocol and translation details.  Many new, innovative, high performance
architectures are possible using Bridges.  A common configuration in PCI PC
systems is shown in figure 1.

(figure not available in text file) 
Figure 1 - A Common PCI System Configuration


In figure 1, a CPU bus is bridged to a PCI bus which in turn is bridged to 
a LegacyI/O bus.  Any CPU or Legacy I/O bus can be supported as long as the 
corresponding Bridge is available.  Bridges in PCI systems are typically 
single chip devices, part of an off the shelf chipset with which a complete 
system can easily be designed.

   The PCI bus can support single chip PCI devices or add-in card 
connectors.  In order to operate at the high speeds that are possible 
with PCI, the PCI Specification defines a limit of 10 loads on any PCI 
bus.  A single chip PCI device is considered to be 1 load and an add-in 
card with connector is counted as 2 loads.  Given this limit, PCI buses 
are typically limited to 3 or 4 add-in card connectors.  With the popularity 
of PCI, it’s easy to need more add-in card slots than are available in a 
system.  Bridges solve this limitation.


PCI-PCI Bridges Expand Connectivity

Bridges may connect PCI buses to PCI buses in which case they are called 
PCI-PCI Bridges or PPB’s for short.  Using a PPB to create another PCI bus 
in the system gives us another 10 loads to add more add-in slots.  PPBs can 
also enable expansion by being placed on add-in cards to create multi-
channel combinations.  Common multi-channel add-in cards currently available 
include SCSI and Ethernet.  PCI allows the addition of up to 256 different 
PCI buses.  Figure 2 shows a typical multiple PCI bus configuration.

(figure not available in text file) 
Figure 2 - A Multiple PCI Bus Configuration


   Additional PCI buses may be nested to any level to form an inverted tree 
topology.  From any given point in the tree there may be a PCI Bus above or 
below you.  Buses above you are said to be Upstream while buses below you 
are said to be Downstream.  A disadvantage to being located on a lower level 
PCI bus is that the latency time to form a connection through all the 
Bridges in the system can become high causing choppy data bursting 
performance.


PCI-PCI Bridges Improve Performance

   While the latency time to connect to a lower level PCI bus may be higher, 
a separate PCI bus can improve overall system performance by off-loading 
processing to local PCI buses.  This reduces bus traffic and increases 
system throughput.  This is a common technique in add-in card architectures 
where local processing off-loads the main CPU.  Two examples of PPB add-in 
card connectivity and performance improvement are shown in figure 3.

 
(figure not available in text file) 
Figure 3


   The AHA-3940 takes advantage of PPB expansion to put 2 SCSI channels 
behind a bridge.  The AHA-3985 takes advantage of PPB local processing to 
perform on board RAID parity calculations.


Supporting PCI-PCI Bridges
   
   For PPBs to work, devices on downstream buses must be accessible by their 
driver software, independent of the bus topology used.  The only distinction 
possible is that the downstream device has a different bus number.  In order 
to do this, the system BIOS firmware must correctly initialize the PCI 
Bridge architecture and the host chipset bridge must support PPB 
architectures.

   On power-up initialization, the system BIOS firmware must correctly 
identify all PCI buses in the system and all PCI devices in the system, 
then assign non-overlapping I/O and Memory resources to everyone.  This 
includes PCI buses and devices on the motherboard as well as add-in cards.  
The PPBs act as gate keepers to all downstream buses and devices.  PPBs 
are programmed with the bus numbers above and below it as well as the I/O 
and Memory resource ranges of all devices behind it.  They pass along bus 
cycles based on the ranges programmed into them.

   A host chipset bridge must be able to correctly pass down PCI 
configuration cycles so that the system BIOS firmware can correctly identify 
all PCI devices in the system.  The host bridge must also follow the access 
ordering rules defined by PCI, otherwise coherency problems and system 
deadlocks can occur.  Details on PPB implementation are found in the "PCI 
to PCI Bridge Architecture Specification v1.0 4/5/94" available from the 
PCI SIG (800) 433-5177, (503) 797-4207


Other Bridge Topologies

   Early PCI systems adapted their existing VL designs to PCI by adding a 
VL to PCI Bridge to the system as show in figure 4.  While not offering 
optimal PCI performance, these systems had the benefit of being able to 
offer VL, PCI and ISA slots for add-in cards.

 
(figure not available in text file) 
Figure 4  VL-PCI System


   To avoid the latency problems of going through several levels of 
bridging, some systems use a peer bus architecture as shown in figure 5.  
One of the peer PCI buses is utilized as shown previously and the other is 
used only for slot connector expansion.  With both PCI buses at the same 
level, access latencies are minimized.  However synchronizing transactions 
on the CPU side become more complex.

 
(figure not available in text file) 
Figure 5  Peer PCI Bus System


Summary

   PCI systems make extensive use of bridging to achieve innovative, high 
performance architectures.  Through bridging, systems may be customized for 
slot expansion.  Bridging on add-in cards can add multiple channels or 
improve performance by off-loading the CPU.  However to take advantage of 
all that bridging can offer, system BIOS firmware and chipset silicon need 
to be able to support bridge operation.