VMWare + Linux With Multiple Network Interfaces

Friday, 23 October 2020

I use a couple of Debian VMs (in VMWare Workstation on a Windows box) for local development; at this point my Macbook just runs Firefox, Slack, VSCode, and iTerm (and is still nowhere near responsive most of the time), using SSH tunnels and VSCode’s SSH extension for actual development.

All the VMs are connected to the network in bridged mode; they each appear on the network as a separate device. The Windows box itself is connected to the rest of my home network via a PCIe wireless card; here’s a quick overview of the entire home network:

With this arrangement, every connection between my laptop and home server needs (at the very least) to go through the closest wireless access point, which can be quite slow. In addition, I have to tether my laptop to the internet via a mobile hotspot when my ISP occasionally has issues. I don’t really want to tether my home server to the hotspot, so in this case the laptop and server are completely cut off from each other.

These two machines are literally sitting next to one another, so what if I just ran Ethernet between them to create a wired connection? Luckily, VMWare allows for multiple network interfaces per VM. Essentially, you can connect the host machine to (say) both a wired and wireless network connection, and expose both to the virtualized OS. The laptop also needs one of these because everything is Thunderbolt-ed now.

Next, it’s easy enough to set up the laptop so it has a static IP on this wired “network”:

Now for the interesting bit. Add a new virtual network to VMWare representing this wired “network”. With VMWare Workstation open, navigate to Edit > Virtual Network Editor, and add a virtual network representing this new wired “network”:

With that set up, add a second network adapter to each individual VM that needs access to this wired “network”, linking it to the virtual network that was created 👆:

At this point (you might have to reboot first), these VMs can (at least theoretically) see my laptop (and vice versa) over the wired connection. Subsequent setup on the virtualized OS is probably required first, though, and the specifics will differ based on OS/distribution; here’s what I did on Debian (11):

# Look for the new network interface's identifier. Here it's `ens37`
$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:0c:29:3b:18:af brd ff:ff:ff:ff:ff:ff
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:0c:29:6e:d3:64 brd ff:ff:ff:ff:ff:ff
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
    link/ether 02:42:d2:ba:1a:17 brd ff:ff:ff:ff:ff:ff

# Add a block identifying this network (+ a static IP) to `/etc/network/interfaces`
$ cat <<HEREDOC | sudo tee -a /etc/network/interfaces
allow-hotplug ens37
iface ens37 inet static
address 10.0.0.11
netmask 255.255.255.0
gateway 10.0.0.1
HEREDOC

# Restart the network interface
$ sudo ifdown ens37 && sudo ifup ens37
# Or the VM
$ sudo reboot

That’s now sufficient to allow my laptop to access the VPN over the wired connection. I’m much more concerned about latency than throughput with this setup, so these numbers seem promising (wireless on the left, wired on the right); it looks like a 3-4x improvement on average:

There’s one more problem with the Debian VM; all internet-bound traffic is also routed to the wired connection (presumably because it specifies an explicit gateway while the wireless connection uses DHCP), so it’s effectively cut off from the WAN:

$ curl -v https://google.com/
*   Trying 74.125.24.139:443...
*   Trying 2404:6800:4003:c03::65:443...
* Immediate connect fail for 2404:6800:4003:c03::65: Network is unreachable
*   Trying 2404:6800:4003:c03::8b:443...
* Immediate connect fail for 2404:6800:4003:c03::8b: Network is unreachable
*   Trying 2404:6800:4003:c03::64:443...
* Immediate connect fail for 2404:6800:4003:c03::64: Network is unreachable
*   Trying 2404:6800:4003:c03::66:443...
* Immediate connect fail for 2404:6800:4003:c03::66: Network is unreachable

# Note the first `default` line; it's routing all traffic to `ens37`, the wired "network"
$ ip route list
default via 10.0.0.1 dev ens37
10.0.0.0/24 dev ens37 proto kernel scope link src 10.0.0.11
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.1.0/24 dev ens33 proto kernel scope link src 192.168.1.110

A fix for this issue needed some Googling: edit the entry for ens33 (the existing wireless network) in /etc/network/interfaces so it looks like:

 allow-hotplug ens33
 iface ens33 inet dhcp
+up /bin/ip route change default via 192.168.1.1 dev ens33

After a VM reboot, all internet-bound traffic should now be routed to the wireless network:

$ curl -v https://google.com/
*   Trying 142.250.71.46:443...
* Connected to google.com (142.250.71.46) port 443 (#0)
(...)

$ ip route list
default via 192.168.1.1 dev ens33
10.0.0.0/24 dev ens37 proto kernel scope link src 10.0.0.11
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.1.0/24 dev ens33 proto kernel scope link src 192.168.1.110
Edit