Here is what I've experienced.  You can forget about a serial mouse for the
ST.  They are not compatable with the ST mouse.  But you can connect a bus
mouse to the ST, I have done it.  I have a Microsoft bus mouse connected to my
ST and it is great.  The following are some instructions I got off one of the
ST ftp sites about that subject and is what I followed to connect the 
Microsoft mouse to my machine.

-----

Here's a repost of my article of wayyyy back (Feb 1989) on how to hook an
Microsoft mouse to an Atari ST.  No software changes are required.  All it
takes is creating a special cable between the mouse and the Atari.

I've been using a Microsoft mouse now for around 2 years on my 1040, with nary
a glitch.  It's so much better than the Atari mouse, I'd never go back. (IMHO)

Well, here is how to connect a Microsoft Mouse to an Atari ST.  (520/1040)
The Atari has a 9-pin D-type connector.  The Microsoft Mouse has a rather
strange connector on the end.  To prevent having to cut the mouse cable up,
I made this adaptor.  If you still can't follow it, please let me know and
I'll try to redescribe it.  This does work for me.  I've been using it for
6 or so months now [Note, it's been 2 years now!] with no problems at all.
It does take some soldering skills, so if you're not so hot at soldering,
find a friend to help.

As far as I know, this does not effect your Atari's warranty.  However, I
have not asked Atari.

Atari ST        Microsoft Mouse                 ST expected signal
----- --        --------- -----                 -- -------- ------
1               5                               XB
2               3                               XA
3               8                               YA
4               9                               YB
5               N/C                             N/C
6               2                               Left button
7               N/C                             +5VDC (N/C)
8*              6 and also tie to pin 1         Ground
9               7                               Right button

* = I am not sure why I have pins 6 & 1 tied together, but it does work.

The Microsoft mouse connector looks like this.  This is the solder side,
with the leads pointing up.

  ---------------
  |     1       |
  |  2 3 4 5    |
  | 6 7 8 9 10  |
  |             |
  ---------------

  If you can not use the connector, the pins are numbered as follows.  This
  is the connector side from the Microsoft Mouse plug:


          /------------\        This is a view at the front of the connector.
         / Blank "KEY"  \       It is where the plug from the mouse would be.
        /----------------\      The pin labeled 'n' is not connected.
        | n   4   X   8  |      The place labeled 'X' is filled, and does not
        | 9   7   1   2  |      have a pin.
        |     5   6      |
         \              /
          \------------/ 


Have fun!

Jeff Beadles            jeff@onion.pdx.com