XenCenter causing a SYN flood on port 3389 in a PVS environment

Had an interesting morning. Our network team discovered my workstation was making around 6000 requests an hour on RDP port 3389 through our internal firewall to an unroutable network reserved in our server subnet. Basically a SYN flood (DOS/Denial of Service attack) was being executed internally from my workstation. I know my machine is clean and the only thing I had open was XenCenter. It had been running for about a week now, that was the last time I rebooted my workstation. So time to put on the detective hat and figure this one out.

While using TCPView to do a live netstat, I discovered that XenCenter by default will always establish a connection via RDP when you click the Console tab. It tests to see if RDP is available on the VM and then ungreys that “Switch to Remote Desktop” option. Even if you are using a console session, XenCenter wants to see if RDP is an available option to you. You know that little flash of the console you usually see after hitting the Console tab? That’s the XenCenter console checking for RDP and “connecting” to the VM transparently to verify RDP is available. Not so transparent and actually annoying but I never thought too much about it.

SYN_SENT on 3389:

Established connection:

Terminating connection:

The problem is in a PVS environment, you usually have your network split between a streaming traffic NIC and a regular traffic NIC. The streaming traffic NIC is supposed to be the first NIC/device. So Device 0. It will look something like this on all your VMs:

Where the first streaming NIC is only routable within the Blade enclosure or server subnet and the secondary NIC routable and used for regular network traffic.

Well the problem is that when you click the Console tab on one of these VMs, XenCenter will send a SYN request to what IP is at Device 0. So in our case, an unroutable IP in the server subnet.

Not a problem right? Well it never stops. It continuously sends the SYN requests attempting to connect. Even if you click off the Console tab or go to another VM, it continues to try RDP on that IP. Our firewall separating workstation and server subnets was getting hammered. You can verify because your Console will have the “Switch to Remote Desktop” option greyed out during this whole process.

and TCPView will show all those little red SYN_Sent attempts. After a few days of leaving XenCenter up and clicking from console to console, the amount of traffic hitting your firewall will be tremendous. It will look like a SYN flood attack. If you have an IPS or IDPS (Intrusion Prevention or Intrusion Detection & Prevention System), it might even shut down your port.

I called Citrix and submitted a ticket with their development team. I got a call back later and there is a work around. In XenCenter, go to Tools > Options > and click the Console option. Then uncheck “Enable Remote Desktop console scanning”:

When you uncheck this, it will also uncheck “Automatically switch to the Remote Desktop console when it becomes available”. This is fine:

After this you will notice the SYN flood will immediately stop and all your VMs will now have the “Switch to Remote Desktop” option ungreyed from the get go. If you click it, then it will attempt the RDP connection and you will see the SYN_Sent again:

In my opinion, Citrix should fix this but stopping the SYN requests after you click away from the Console tab. This is not an issue that will impact many people but if you are running a PVS environment and you have it setup using Citrix best practices with 2 NICs and the streaming NIC is not accessible from your workstation subnet, you will eventually run into this issue. The longer you keep XenCenter open, the worse it will get.

If I get any updates from Citrix on a fix, I will post here. For now the work around will work fine. You just won’t have the Remote Desktop automatic console scanning available for your regular environments anymore. Not really a big loss for me but it might be for you depending on your environment.