Running Citrix Gateway/NetScaler (unsupported) on Google Cloud Platform

During a recent project, I needed to get a full Citrix XenDesktop environment up and running. While the Citrix team provides a nice demo environment for partners, it became necessary for us to have an environment we could customize (and we love to tinker).

So, today, it looks like we have 2 options:

Deploy officially leveraging Citrix Cloud. This leverages a mixture of Citrix Cloud to provide front-end & NetScaler services, with XenApp on Google Cloud Platform offering back-end services.

GCP Integrated with Citrix Cloud

Install on-premise Citrix XenDesktop on GCP. In this scenario, we’ll need all the elements running in GCP, including NetScaler to provide remote access. Bad news: NetScaler doesn’t run on GCP, so we’ll need to figure this out.

Which NetScaler is Right for Me?

So now we’ve committed to running NetScaler on GCP to give us maximum control. Best-case scenario, we would just install it from the Google Cloud Launcher, but it’s not available there today. According to the Citrix NetScaler documentation, supported hypervisors include AWS, Azure, ESXi, KVM, and XenServer.

Woohoo! I see KVM on the list and I know GCP leverages a customized version of KVM. Unfortunately, further digging reveals that the customization of the VM required to run NetScaler won’t allow it to run directly on GCP as a VM. Sigh…

But…all is not lost! GCP now offers nested virtualization, which supports running a custom hypervisor on Compute Engine. With this, I’ll be able to tweak KVM to support NetScaler.

Building the Virtualization Node

Before getting started, it’s worth noting the documented restrictions. Most importantly, this is only supported on Linux & KVM, so no Windows or Hyper-V.

I’m an Ubuntu guy, so I decided to start with Ubuntu 17.10. The steps to enable virtualization requires you first build a custom image that enables it. From there, you create a VM, with the custom image, and you now have a server that can support virtualization.

Once booted, we need to get KVM installed. Here’s all the packages I used for Ubuntu.

sudo apt-get install libvirt-dev xsltproc libxml2-dev libxml2 virt-manager qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils

With KVM installed, we can do a quick pre-flight to make sure KVM is detecting hardware virtualization.

mike@ns-west-1:~$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Optionally, I found it much easier to use the virt-manager GUI app to build VMs, which requires I can VNC in to my server. Make sure to open the VNC port (5901) on your VPC firewall.

sudo apt-get install gnome-core gnome-panel vnc4server
$ vncserver
You will require a password to access your desktops.
$ vncserver -kill :1

Installing & Configuring NetScaler

For the installation, we’re going to run NetScaler-on-a-stick. This essentially means the NetScaler will have it’s own subnet that all the interfaces will reside on, and we’ll use VPC routing to get traffic over to the NetScaler.

The first thing I would do is to knock down the existing iptables firewall rules. This will ensure existing network restrictions on the VM aren’t going to impact the NetScaler.

sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT
sudo iptables -t nat -F
sudo iptables -t mangle -F
sudo iptables -F
sudo iptables -X

KVM installs a default subnet on the VM, 192.168.122.0/24. In this example I will use this default network.

mike@ns-west-1:~$ ifconfig virbr0
virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:74:3f:bd  txqueuelen 1000  (Ethernet)
        RX packets 77603397  bytes 62697064390 (62.6 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 54139811  bytes 6981400583 (6.9 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

We’ll need to make sure we build a VPC route to send traffic destined to 192.168.122.0/24 to our VM.

VPC Route to NetScaler

Now that we have our environment ready, we can install NetScaler. You will have to download the VPX image off of citrix.com. If you’re going in via VNC and using the virt-manager UI, you can follow this guide. The end result is this massive qemu command

qemu-system-x86_64 -enable-kvm -name guest=freebsd6.0,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-freebsd6.0/master-key.aes -machine pc-i440fx-artful,accel=kvm,usb=off,vmport=off,dump-guest-core=off -cpu Westmere -m 2048 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 85b530f2-ada7-406b-9091-be5c2f3ddcb2 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-freebsd6.0/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/home/mike_wilusz/NSVPX-KVM-12.0-56.20_nc_32.qcow2,format=qcow2,if=none,id=drive-ide0-0-0 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=25,id=hostnet0,vhost=on,vhostfd=27 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:85:02:71,bus=pci.0,addr=0x3 -netdev tap,fd=28,id=hostnet1,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:d3:88:19,bus=pci.0,addr=0x8 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -vnc 127.0.0.1:0 -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=1 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on

After NetScaler boots, you’ll need to give it a basic network configuration to be able to access it. You will enter these commands on the NetScaler console.

set ns config -ipaddress 192.168.122.XXX -netmask 255.255.255.0   
add route 0.0.0.0 0.0.0.0 192.168.122.1 
save ns config  
reboot

You should now be able to access the web UI via another VM in the VPC. We have not set up any external connectivity to the NS yet. From here, the configurations will vary in most environments.

The final tip I’ll provide is how to open external access. In a Citrix XenDesktop installation, it is required to open the VIP (virtual IP) to the outside world, and this address is what external users will use to access Citrix XenDesktop remotely. In the below configuration, we’re going to forward all SSL traffic sent to the Ubuntu VM (10.138.0.2 in this example) to the NetScaler VIP (192.168.122.150 in this example). This means that the external IP address for the VM will have all SSL traffic forwarded to the VIP on NetScaler.

sudo iptables -t nat -A PREROUTING -p tcp -d 10.138.0.2 --dport 443 -j DNAT --to-destination 192.168.122.150:443
sudo iptables -t nat -A POSTROUTING -d 192.168.122.150 -j MASQUERADE
sudo iptables -A INPUT -p udp -j ACCEPT
sudo iptables -A FORWARD -p tcp -j ACCEPT
sudo iptables -A OUTPUT -p tcp -j ACCEPT
sudo iptables -A OUTPUT -p udp -j ACCEPT