Best hypervisor for routers

r00igev@@r

r00igev@@r

Honorary Master
Joined
Dec 14, 2009
Messages
15,640
Reaction score
14,157
Location
Draadloos Bantha poo doo in 4ways
I'm struggling to find a hypervisor that doesn't suck golf balls running an x86 router. Bare Metal works great but the drop in performance to virtual is dramatic.

Any good ideas???
I've tried cockpit-machines and xcp-ng.
 
Used Hyper-V and Proxmox. Getting same throughput as dedicated router. Still want to play with ESXI. What router software are you running?
 
I've used pfSense and opnsense under VMware/Hyper-V/Proxmox - performance was never an issue.
Always used decent hardware and only Intel network cards.
 
Problem I had with OPNsense is that there is an issue with the hardened BSD networking where you can't get much better than about 400Mbps throughput. They will only address this next year when they switch away from the hardened BSD.
 
I used to run pfSense and OPNsense on kvm, using a bridged setup on Intel cards. The cards were setup on the client with paravirtualization. You have to disable hardware checksum offloading on the client, but I always managed to hit 1Gbps, basically maxing out the links. Also, make sure cpu hardware crypto is working (AES-NI extensions). When using pf/opnSense as a SSL terminator, I can still hit 1Gbps.
 
Nuro said:
I used to run pfSense and OPNsense on kvm, using a bridged setup on Intel cards. The cards were setup on the client with paravirtualization. You have to disable hardware checksum offloading on the client, but I always managed to hit 1Gbps, basically maxing out the links. Also, make sure cpu hardware crypto is working (AES-NI extensions). When using pf/opnSense as a SSL terminator, I can still hit 1Gbps.
Click to expand...
I see offloading was enabled in the vm. Let me try that!
 
r00igev@@r said:
I'm struggling to find a hypervisor that doesn't suck golf balls running an x86 router. Bare Metal works great but the drop in performance to virtual is dramatic.

Any good ideas???
I've tried cockpit-machines and xcp-ng.
Click to expand...

I'm testing a sophos-xg firewall vm on vmware workstation with 2 10gbpe sfp+ cards. No issues since the virtual adapters are both VMXNET3.

Considering a complete virtual based environment like this inside vshphere. Even ports forwarded to the vm while the ESXI servers have some creative natting to disallow the hardware machines to be public facing though the physical nic.
 
KVM, ESXI, Proxmox, hyper-V, hell, I've even managed to get good performance through an Oracle Virtualbox instance.

Obviously if you're looking at hitting 10G and the like, KVM / ESX are gonna be your best options along with some offload-capable NIC.
 
Maybe try disabling GRO, ECN on the host adaptors - "could" help
 
InvisibleJim said:
Any particular config you used that got it working? Also curious as to what router software/distro you are using if you'd care to share.
Click to expand...
I used CHR and also tested the latest debian buster with the kernel updated using backports. On the later I used ssl tunnels that leverage aes. Also the intel ethernet controllers help reduce CPU load.
 
Hyper-V has some kuk that is a Microsoft legacy to get things work.

The free headless hypervisor is a waste of time. Wasted a day of my life on that before going standard server with desktop experience. There are two things that don't work well on cli. Manging firewall rules (or hypervisors).
 
r00igev@@r said:
Hyper-V has some kuk that is a Microsoft legacy to get things work.

The free headless hypervisor is a waste of time. Wasted a day of my life on that before going standard server with desktop experience. There are two things that don't work well on cli. Manging firewall rules (or hypervisors).
Click to expand...

Hyper-V core works well when you have a bunch of machines running in an AD. We've used it extensively across our infrastructure with nodes physically spread over 5 or so DCs. Bare metal to a booted HyperV core server takes a few minutes to get up and running, and seen upwards of 500 days uptime on nodes without hassles... The CLI is rubbish and designed as a torture tool - but a necessary evil for mastering the network stack.

In terms of running CHR; Hyper V has probably the best network stack out there in terms of performance (tested Proxmox, KVM, VMWare and Xen). Only issue was actual peak packet (PPS) throughput - high levels break everything. There is a bunch you can do to refine the NIC<>CPU relationship to improve this, but we turned to iron boxes in this regard, though FPGA looks promising. We host/use many CHRs on 10G HyperV core nodes for a bunch of tasks - from L2TP tunnel brokers to firewalls, management and monitoring. With the added advantage of HA (both via HyperV's inbuilt failover cluster (read: headaches) and replication, and using something like Veeam, you get service resiliency with little hassle). $40 odd for a perpetual 1G license is the win with CHR.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X