VRFs on Linux

VRF (Virtual Routing and Forwarding) allows having multiple and separated routing tables on the same system.

On Linux, VRF support has been started on 4.3 kernels. Ubuntu 16.04 brings 4.4 kernels but mind that 4.5 kernels have some important patches too.

This post will show how to create two different VRFs, one dedicated to a virtual bridge. On each virtual bridge, a virtual router is running. The VM used for the tests below is a Ubuntu 16,.04 installed with EVE-NG.

Create a VRF

Two VRFs will be created:

ip link add red type vrf table 1
ip link add green type vrf table 2

Don’t forget to bring both up:

ip link set dev red up
ip link set dev green up

# ip -br link show type vrf
red              UNKNOWN        9a:ca:96:75:f8:f5 <NOARP,MASTER,UP,LOWER_UP>
green            UNKNOWN        8e:b6:6f:25:64:10 <NOARP,MASTER,UP,LOWER_UP>

Assign a Network Interface to a VRF

An EVE-NG switch will be assigned to each VRF:

ip link set vnet0_1 master red
ip link set vnet0_2 master green

# ip link show vnet0_1
3: vnet0_1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue master red state UP mode DEFAULT group default qlen 1000
    link/ether ae:81:73:24:b5:66 brd ff:ff:ff:ff:ff:ff
# ip link show vnet0_2
8: vnet0_2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue master green state UP mode DEFAULT group default qlen 1000
    link/ether 0a:9d:e0:76:8b:84 brd ff:ff:ff:ff:ff:ff

Set FIB rules

Prior to the v4.8 kernel, some additional rules must be added to properly forward traffic on interfaces:

ip rule add iif red table 1
ip rule add oif red table 1
ip rule add iif green table 2
ip rule add oif green table 2

# ip rule show
0:      from all lookup local
32762:  from all oif green lookup 2
32763:  from all iif green lookup 2
32764:  from all oif red lookup 1
32765:  from all iif red lookup 1
32766:  from all lookup main
32767:  from all lookup default

IP configuration

An IP address and a default route will be assigned to each virtual bridge and VRF:

ip addr add 10.254.0.1/24 dev vnet0_1
ip addr add 10.254.0.1/24 dev vnet0_2
ip route add 0.0.0.0/0 via 10.254.0.2 table 1
ip route add 0.0.0.0/0 via 10.254.0.2 table 2

Connectivity test

Both virtual routers connected to virtual bridges are configured as follows:

R1: e0/0:10.254.0.2/24 lo0:1.1.1.1/32 . R2: e0/0:10.254.0.2/24 lo0:2.2.2.2/32

Remember to bind ping (and traceroute) to an interface assigned to the VRF:

# ping -I vnet0_1 -c5 10.254.0.2
PING 10.254.0.2 (10.254.0.2) from 10.254.0.1 vnet0_1: 56(84) bytes of data.
64 bytes from 10.254.0.2: icmp_seq=1 ttl=255 time=0.434 ms
[...]

# ping -I vnet0_2 -c5 10.254.0.2
PING 10.254.0.2 (10.254.0.2) from 10.254.0.1 vnet0_2: 56(84) bytes of data.
64 bytes from 10.254.0.2: icmp_seq=1 ttl=255 time=0.649 ms
[...]

Local networks are reachable, now test the routing:

# ping -I vnet0_1 -c5 1.1.1.1
PING 1.1.1.1 (1.1.1.1) from 10.254.0.1 vnet0_1: 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=255 time=0.649 ms
[...]

# ping -I vnet0_2 -c5 2.2.2.2
PING 2.2.2.2 (2.2.2.2) from 10.254.0.1 vnet0_2: 56(84) bytes of data.
64 bytes from 2.2.2.2: icmp_seq=1 ttl=255 time=0.452 ms
[...]

Current limits

VRF support on Linux is quite new, so there are many limits:

most of the applications cannot be VRF aware, even if local services can listen at all VRF (sysctl -w net.ipv4.tcp_l3mdev_accept=1 with 4.5 kernels);
there is no way to start a client command (telnet, ftp, ssh, …) using a specific VRF.

Edit: currently a command can be executed and “attached” to a specific VRF using: ip vrf exec blue ping 8.8.8.8