I’ve created a git repository for the OpenStack nova ports with my changes. Please note, it’s still experimental, in development and not intended for production use.
Next week I’ll write a manual how to get it work, configure networks, floating IPs, set glance metadata to use HV instead of PV, etc.
Currently I’m working on neutron and automation of that stuff. I’m going to setup a dedicated build server and try to run FreeBSD/Xen inside ESXi or XenServer (nested virtualization). It’s really boring to deploy baremetal and rebuild everything 3 times a day :)
I’m also developing a Continuous Delivery solution, progress has slowed down but I will resume all activities soon. It would be an artifact management solution with integrated monitoring and deployment features.
- Virtualization using Xen hypervisor on FreeBSD 11.0
- Basic networking with legacy nova-network
- Internet connectivity from instances (SNAT)
- Floating IP addresses (DNAT)
- COW (Copy-On-Write) images
- Investigate issue with XENBUS
- Migrations and snapshots
- Security Groups
- Replace linux specific things like losetup, kpartx
- Look at os-vif, os-brick to mount cinder volumes
- bhyve (actually I got it working but booting linux images is not user friendly, requires a grub loader, providing grub config, etc..)
- Tests! and devstack
It would be nice to add jails support into nova but I’m aware because on FreeBSD wiki page CPU+RAM limits for jails are marked as “Not fully working / stalled”
Also due to missing implementation of some functions in libvirt qemu driver it’s impossible to run nova on freebsd/qemu without removing NUMA introspection features in nova.
However when using Xen and PVHVM drivers (qemu) you may experience a bug with libvirt/xen which consists in that Linux guest is missing XENBUS and you have to wait 300 seconds. So I added an extra option “force_xen_phy” to force using of phy driver (it works only with raw images without COW).
I plan to stabilize nova soon, and get a PoC version of neutron bridge/OVS driver and Cinder ZFS driver in December. Specs, upstream proposal and contribution scheduled for February.
For the packet filtering PF firewall is used. Every rule is placed to appropriate anchor (sub-ruleset).
You need to add these rules into pf.conf
and nova services will create required sub-rule sets.
Interesting, I couldn’t get PF to process packets originated from the localhost (nova) or from the instance interface itself to the floating IP but it works with metadata service by routing packets again into loopback.
pass out route-to (lo0 127.0.0.1) inet proto tcp from any to 169.254.169.254 port = http flags S/SA keep state
I found that
PF can not redirect traffic that originates from the host itself because the routing decision for the traffic has been already made by the time it gets to the filter. It’s a FreeBSD specific limitation that does not exist on OpenBSD’s PF for example.
I also added several rc scripts for nova services.