Reply From Interface – Policy Based Routing

I ran into a problem recently on one of my Debian based servers that has a relatively complex networking setup. Specifically, the server connects to two VPNs – network A and network B – and clients from network B need to be able to access services served from interface A of the server. I could go through the trouble of having DNS overrides that point the network B clients to interface B and just serve the content on both interfaces, but I had a few reasons why that wasn’t an ideal solution.

However, because the server has an address in both networks, when it would receive packets on interface A from a network B address it would automatically try to reply from interface B, since that’s the route that was available for network B. I couldn’t change the routing table without breaking access for network B clients to the services hosted on interface B.

I knew the answer was going to be policy based routing, but I wasn’t 100% sure exactly how to set it up. In the end it was rather complex and not quite as intuitive as one would think. Here’s how it worked:

Create a Custom Routing Table

To start, create a custom routing table. This is the routing table that will be used instead of the default routing table, which will say that the route to network B is available via interface A.

echo "200 custom" > /etc/iproute2/rt_tables

The routing table needs to have a route to network B. In our case a default route works fine, but other routes may be required.

ip route add default via <network_A_gw> dev <interface> table custom
ip -6 route add default via <network_A_gw6> dev <interface> table custom

Create the Routing Rules

In my situation, I wanted any traffic that is destined for network B but has a source address of interface A to use the routing table that says network B is available via interface A. To do that we add some ip rules.

ip rule add from <interface_A_addr> to <network_B_sub> table custom
ip -6 rule add from <interface_A_addr6> to <network_B_sub6> table custom

This rule says exactly what we want – that any traffic from the interface A address that’s destined for network B, should use the custom routing table that says network B is available via interface A.

Celebrate

And that’s it! With those rules in place, your server will reply via interface A to network B clients that reach out to interface A, but will still be available on interface B for network B clients to reach directly.