WindowsXP comes with a builtin "Internet Connection Firewall". It can 
sometimes get activated when you install other networking applications,
like a VPN client. When ICF is active, you won't be able to connect to
a VNC Server on your WinXP machine, though you will be able to start a
VNC Viewer on the WinXP machine to connect somewhere else.

Here's a link from Microsoft of how to turn it off:

http://www.microsoft.com/windowsxp/pro/using/howto/networking/icf.asp


If you don't have time or authority to go that route, have a look here:
How can I connect to a VNC Server that's behind an unconfigured firewall?

If you've installed the recently released "Service Pack 2" (SP2) for
Windows XP, you have automatically installed a new firewall on your
PC that might be blocking VNC connections. To create an "exception"
for VNC, follow these instructions:

   1. Click Start, click Run, type Wscui.cpl, and then click OK to open 
      Windows Firewall.
   2. Click the Exceptions tab, and de-select the "Don't Allow Exceptions"
      radio-button.
   3. Click Add Port to display the Add a Port dialog box.
   4. Enter port number "5900" or whatever port you've got VNC on that
      PC to listen on (port number = 5900 + "VNC Display Number").
   5. Select the TCP protocol.
   6. In the Name field, type "VNC".
   7. Click Change Scope to view or to set the scope for the port 
      exception, and then click OK.
   8. Click OK to close the Add a Port dialog box.
   
That should do it! More info here:

http://support.microsoft.com/default.aspx?kbid=875357#7

Another reason you might have difficulty connecting to a WinXP
machine is due to incomptability between the VNC Server and some 
special features of WinXP (see below for details). If you're 
having trouble connecting to a VNC Server running on WindowsXP, 
try one (or more) of these:

First, check that you haven't logged on as several different users
simultaneously. If you have (at any point since the machine booted), 
go to Task Manager->Users and check the "ID" of the Active session.  
If it's not zero then WinVNC won't work -- the "ID" of the Active 
session must be 0. Try connecting with VNC after you've rebooted
the WinXP machine.

Next, try disabling "Fast User Switching". WinXP uses Terminal Services
to implement Fast User Switching, and this Terminal Services isn't
compatible with the current versions of WinVNC.

Next, try disabling "Remote Desktop". That also uses Terminal Services,
and will be problematic as above.

Lastly, try disabling "Use the Welcome Screen". WinXP doesn't use
Terminal Services for the Welcome Screen, but some VNC users have
indicated that it helped them connect. Your mileage may vary.

At a high-level, here's what's going on: VNC can only remote a WinXP
console session when the session is "Active".  And the console in 
Windows XP is treated rather like a Terminal Services Client that by 
default is connected to the special "session zero".  Session zero is 
where all services run, so it's the one that the VNC service remotes. 
If you disconnect "session zero" somehow, say by using fast user 
switching or Remote Desktop, then it becomes unavailable via VNC. 
Thanks to "Wez" at RealVNC for the explanation!