I ran into a problem recently on one of my Debian based servers that has a relatively complex networking setup. Specifically, the server connects to two VPNs – network A and network B – and clients from network B need to be able to access services served from interface A of the server. I could go through the trouble of having DNS overrides that point the network B clients to interface B and just serve the content on both interfaces, but I had a few reasons why that wasn’t an ideal solution.
However, because the server has an address in both networks, when it would receive packets on interface A from a network B address it would automatically try to reply from interface B, since that’s the route that was available for network B. I couldn’t change the routing table without breaking access for network B clients to the services hosted on interface B.
I knew the answer was going to be policy based routing, but I wasn’t 100% sure exactly how to set it up. In the end it was rather complex and not quite as intuitive as one would think. Here’s how it worked:
Create a Custom Routing Table
To start, create a custom routing table. This is the routing table that will be used instead of the default routing table, which will say that the route to network B is available via interface A.
echo "200 custom" > /etc/iproute2/rt_tables
The routing table needs to have a route to network B. In our case a default route works fine, but other routes may be required.
ip route add default via <network_A_gw> dev <interface> table custom
ip -6 route add default via <network_A_gw6> dev <interface> table custom
Create the Routing Rules
In my situation, I wanted any traffic that is destined for network B but has a source address of interface A to use the routing table that says network B is available via interface A. To do that we add some ip rules.
ip rule add from <interface_A_addr> to <network_B_sub> table custom
ip -6 rule add from <interface_A_addr6> to <network_B_sub6> table custom
This rule says exactly what we want – that any traffic from the interface A address that’s destined for network B, should use the custom routing table that says network B is available via interface A.
Celebrate
And that’s it! With those rules in place, your server will reply via interface A to network B clients that reach out to interface A, but will still be available on interface B for network B clients to reach directly.