Shell Script to Route Multiple Public/Private Interfaces

If you have a server with multiple interfaces – either public and/or private – your routing table might look something like this:


sh# ip route list
default via 17.10.20.1 dev eth1  metric 100 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.51 
17.10.20.0/23 dev eth1  proto kernel  scope link  src 17.10.20.51 
105.104.72.16/28 dev eth2  proto kernel  scope link  src 105.104.72.23

sh# ip route list

default via 17.10.20.1 dev eth1 metric 100

192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.51

17.10.20.0/23 dev eth1 proto kernel scope link src 17.10.20.51

105.104.72.16/28 dev eth2 proto kernel scope link src 105.104.72.23

This example shows one private interface with IP 192.168.0.51, two public interfaces with IPs 17.10.20.51 and 105.104.72.23, and a default route to 17.10.20.1. This means that any traffic to/from an IP outside the interface’s subnets is sent to 17.10.20.1 — and this is where problems occur (and probably why you’re reading this article). ;-)

In this example, you have two public interfaces, each (in theory) capable of receiving traffic from the internet, but any IP falling outside of their subnets is sent to 17.10.20.1. For example, a client connects from IP 1.2.3.4 to IP 105.104.72.23 on this server. Since 1.2.3.4 is not within subnet 105.104.72.16/28, the reply is sent to 17.10.20.1, *which is on the wrong interface*. If you want to route each interface properly (sending traffic back out the way it came), you have to setup multiple routing tables, each with it’s own default gateway.

Using the same example, you might create three routing tables; an “intern” table for eth0, a “prov_1” table for eth1, and a “prov_2” table for eth2. Here’s what the creation of those tables might look like:


sh# ip rule add to 17.10.20.0/23 table prov_1 prio 5000
sh# ip rule add to 105.104.72.16/28 table prov_2 prio 2000
sh# ip rule add to 192.168.0.0/24 table intern prio 1000
sh# ip rule add from 17.10.20.0/23 table prov_1 prio 5000
sh# ip rule add from 105.104.72.16/28 table prov_2 prio 2000
sh# ip rule add from 192.168.0.0/24 table intern prio 1000
sh# ip rule list
0:      from all lookup local 
1000:   from all to 192.168.0.0/24 lookup intern 
1000:   from 192.168.0.0/24 lookup intern 
2000:   from all to 105.104.72.16/28 lookup prov_2 
2000:   from 105.104.72.16/28 lookup prov_2
5000:   from all to 17.10.20.0/23 lookup prov_1 
5000:   from 17.10.20.0/23 lookup prov_1
32766:  from all lookup main 
32767:  from all lookup default

sh# ip rule add to 17.10.20.0/23 table prov_1 prio 5000

sh# ip rule add to 105.104.72.16/28 table prov_2 prio 2000

sh# ip rule add to 192.168.0.0/24 table intern prio 1000

sh# ip rule add from 17.10.20.0/23 table prov_1 prio 5000

sh# ip rule add from 105.104.72.16/28 table prov_2 prio 2000

sh# ip rule add from 192.168.0.0/24 table intern prio 1000

sh# ip rule list

0: from all lookup local

1000: from all to 192.168.0.0/24 lookup intern

1000: from 192.168.0.0/24 lookup intern

2000: from all to 105.104.72.16/28 lookup prov_2

2000: from 105.104.72.16/28 lookup prov_2

5000: from all to 17.10.20.0/23 lookup prov_1

5000: from 17.10.20.0/23 lookup prov_1

32766: from all lookup main

32767: from all lookup default

With these rules in place, network traffic from a client with IP 1.2.3.4, connecting to IP 105.104.72.23, will be sent to the “prov_2” table, which will have the proper default gateway for that interface. The result is that traffic coming in on one interface will go back out on the same interface, to the correct default gateway IP.

Using 105.104.72.23 and “prov_2” as an example, the routing table itself might look like this:


sh# ip route add default via 105.104.72.17 dev eth2 metric 100 table prov_2
sh# ip route add 105.104.72.16/28 dev eth2 scope link src 105.104.72.23 table prov_2
sh# ip route list table prov_2
default via 105.104.72.17 dev eth2  metric 100 
105.104.72.16/28 dev eth2  scope link  src 105.104.72.23

sh# ip route add default via 105.104.72.17 dev eth2 metric 100 table prov_2

sh# ip route add 105.104.72.16/28 dev eth2 scope link src 105.104.72.23 table prov_2

sh# ip route list table prov_2

default via 105.104.72.17 dev eth2 metric 100

105.104.72.16/28 dev eth2 scope link src 105.104.72.23

If you have just one server with multiple interfaces, and those interfaces and subnets never change, then you could simply add these ip rule/route commands to a startup script (right after networking starts), or you could use the following script. It will create the proper rules and routes for multiple networks, no matter which interface they are on (including virtual interfaces).

The script depends on /bin/bash and the ipcalc binary (available on most, if not all distributions). The top-most section of the script must be configured for your network.


# --- START OF CONFIGURATION SECTION ---

rt_tables[100]="intern"
rt_tables[200]="prov_1"
rt_tables[300]="prov_2"

NetInfo () {
    # ------------------------------gateway---------metric--table---prio----fwd'ing
    case "$1" in
        17.10.20.0/23)      echo    17.10.20.1      100     prov_1  5000    1 ;;
        84.70.129.64/28)    echo    84.70.129.65    100     prov_2  4000    1 ;;
        105.104.78.152/29)  echo    105.104.78.153  100     prov_2  3000    1 ;;
        105.104.72.16/28)   echo    105.104.72.17   100     prov_2  2000    1 ;;
        192.168.0.0/24)     echo    192.168.0.1     100     intern  1100    0 ;;
        10.201.50.0/24)     echo    10.201.50.1     200     intern  1200    0 ;;
    esac
}

# --- END OF CONFIGURATION SECTION ---

# --- START OF CONFIGURATION SECTION ---

rt_tables[100]="intern"

rt_tables[200]="prov_1"

rt_tables[300]="prov_2"

NetInfo () {

# ------------------------------gateway---------metric--table---prio----fwd'ing

case "$1" in

17.10.20.0/23) echo 17.10.20.1 100 prov_1 5000 1 ;;

84.70.129.64/28) echo 84.70.129.65 100 prov_2 4000 1 ;;

105.104.78.152/29) echo 105.104.78.153 100 prov_2 3000 1 ;;

105.104.72.16/28) echo 105.104.72.17 100 prov_2 2000 1 ;;

192.168.0.0/24) echo 192.168.0.1 100 intern 1100 0 ;;

10.201.50.0/24) echo 10.201.50.1 200 intern 1200 0 ;;

esac

}

# --- END OF CONFIGURATION SECTION ---

The configuration is fairly self-explanatory — an rt_tables array defines the routing table numbers and names used in the /etc/iproute2/rt_tables file, each case statement line defines the gateway for that network/netmask, its metric (in case you use multiple default gateways in the same table), the table name, the rule priority (see the ip rule list command), and if forwarding should be turned on or off for that interface.

Download the script here, or click on the “Copy plain Code” or “Open Code in New Window” symbol to copy-paste the script.


#!/bin/bash
#
# Copyright 2012 - Jean-Sebastien Morisset - https://surniaulula.com/
#
# This script is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free Software
# Foundation; either version 3 of the License, or (at your option) any later
# version.
#
# This script is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
# details at http://www.gnu.org/licenses/.
#
# --- START OF CONFIGURATION SECTION ---

rt_tables[100]="intern"
rt_tables[200]="prov_1"
rt_tables[300]="prov_2"

NetInfo () {
	# ------------------------------gateway---------metric--table---prio----fwd'ing
	case "$1" in
        17.10.20.0/23)      echo    17.10.20.1      100     prov_1  5000    1 ;;
        84.70.129.64/28)    echo    84.70.129.65    100     prov_2  4000    1 ;;
        105.104.78.152/29)  echo    105.104.78.153  100     prov_2  3000    1 ;;
        105.104.72.16/28)   echo    105.104.72.17   100     prov_2  2000    1 ;;
        192.168.0.0/24)     echo    192.168.0.1     100     intern  1100    0 ;;
        10.201.50.0/24)     echo    10.201.50.1     200     intern  1200    0 ;;
	esac
}

# --- END OF CONFIGURATION SECTION ---

RouteNet () {
	# return if we don't have all 8 arguments
	[ "${#@}" != 8 ] && return

	# name the arguments to keep things clear
	dev="$1" ip="$2" net_mask="$3" gw="$4" metric="$5" table="$6" prio="$7" fwd="$8"

	echo sysctl -w net.ipv4.conf.$dev.forwarding=$fwd

	for action in del add
	do
		cat <<EOF
		ip route $action $net_mask dev $dev scope link src $ip table $table
		ip route $action default via $gw dev $dev metric $metric table $table
		ip rule $action from $net_mask table $table prio $prio
		ip rule $action to $net_mask table $table prio $prio
EOF
	done
}

ShowTables () {
	echo -e "\n$1 RULES"
	ip rule list
	for rt in main "${rt_tables[@]}"
	do
		echo -e "\n$1 TABLE $rt"
		ip route show table $rt
	done
	echo ""
}

CheckDeps () {
	if [ -z "`which ip`" ]
	then
		echo "\"ip\" binary not found!" >/dev/stderr
		exit 1
	elif [ -z "`which ipcalc`" ]
	then
		echo "\"ipcalc\" binary not found!" >/dev/stderr
		exit 1
	elif [ -z "`ip rule list|grep 'from all lookup main'`" ]
	then
		echo "kernel does not appear to support policy routing" >/dev/stderr
		exit 1
	fi
}

# parse the command line, skipping stuff we don't know, until there's nothing left
while :
do
	for arg in "$@"
        do
                case $arg in
                        -t) no_change="1"; shift 1;;
                        *)  shift 1 ;;
                esac
                continue 2
        done
        break
done

CheckDeps

echo ""
[ -z "$no_change" ] && \
	cat /dev/null > /etc/iproute2/rt_tables

for rt in "${!rt_tables[@]}"
do
	echo "adding routing table: $rt ${rt_tables[$rt]}" 
	[ -z "$no_change" ] && \
		echo "$rt ${rt_tables[$rt]}" >> /etc/iproute2/rt_tables
done

interfaces="`ip addr show | sed -n 's/^[0-9][0-9]*: \([^:]*\): .*$/\1/p'`"

# reset the command-line arguments with the interface names
set -- $interfaces

ShowTables "BEFORE"

echo "EXEC RULES AND ROUTES"
{
	for dev in "$@"
	do
		for ip_mask in `ip addr show $dev | sed -n 's/^\s*inet \([0-9\.\/]*\) .*$/\1/p'`
		do
			ip=${ip_mask%/*}
			mask=${ip_mask#*/}
			[ -z "$ip" -o -z "$mask" ] && echo "error reading \"ip addr show $dev\" info" >/dev/stderr

			[ "$mask" -eq "32" ] && net="$ip" || \
				net="`ipcalc -n $ip/$mask|sed -n 's/^network[=:][[:space:]]*\([0-9\.]*\).*$/\1/ip'`"
			[ -z "$net" ] && echo "error reading \"ipcalc -n $ip/$mask\" output" >/dev/stderr

			RouteNet $dev $ip $net/$mask `NetInfo $net/$mask`
		done
	done

# remove possible duplicate default routes etc from multiple ips on same network
# sort reverse to exec the deletes first, then the adds
} | sort -u -r | while read line
do
	echo "$line"
	[ -z "$no_change" ] && \
		eval $line >/dev/null 2>&1
done

ShowTables "AFTER"

ip route flush cache

echo "FORWARDING"
sysctl -a 2>/dev/null |  grep '^net\.ipv4\..*\.forwarding'
echo ""

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

#!/bin/bash

# This script is free software; you can redistribute it and/or modify it under

# the terms of the GNU General Public License as published by the Free Software

# Foundation; either version 3 of the License, or (at your option) any later

# version.

# This script is distributed in the hope that it will be useful, but WITHOUT

# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS

# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more

# details at http://www.gnu.org/licenses/.

# --- START OF CONFIGURATION SECTION ---

rt_tables[100]="intern"

rt_tables[200]="prov_1"

rt_tables[300]="prov_2"

NetInfo () {

# ------------------------------gateway---------metric--table---prio----fwd'ing

case "$1" in

17.10.20.0/23) echo 17.10.20.1 100 prov_1 5000 1 ;;

84.70.129.64/28) echo 84.70.129.65 100 prov_2 4000 1 ;;

105.104.78.152/29) echo 105.104.78.153 100 prov_2 3000 1 ;;

105.104.72.16/28) echo 105.104.72.17 100 prov_2 2000 1 ;;

192.168.0.0/24) echo 192.168.0.1 100 intern 1100 0 ;;

10.201.50.0/24) echo 10.201.50.1 200 intern 1200 0 ;;

esac

}

# --- END OF CONFIGURATION SECTION ---

RouteNet () {

# return if we don't have all 8 arguments

[ "${#@}" != 8 ] && return

# name the arguments to keep things clear

dev="$1" ip="$2" net_mask="$3" gw="$4" metric="$5" table="$6" prio="$7" fwd="$8"

echo sysctl -w net.ipv4.conf.$dev.forwarding=$fwd

for action in del add

cat <<EOF

ip route $action $net_mask dev $dev scope link src $ip table $table

ip route $action default via $gw dev $dev metric $metric table $table

ip rule $action from $net_mask table $table prio $prio

ip rule $action to $net_mask table $table prio $prio

EOF

done

}

ShowTables () {

echo -e "\n$1 RULES"

ip rule list

for rt in main "${rt_tables[@]}"

echo -e "\n$1 TABLE $rt"

ip route show table $rt

done

echo ""

}

CheckDeps () {

if [ -z "`which ip`" ]

then

echo "\"ip\" binary not found!" >/dev/stderr

exit 1

elif [ -z "`which ipcalc`" ]

then

echo "\"ipcalc\" binary not found!" >/dev/stderr

exit 1

elif [ -z "`ip rule list|grep 'from all lookup main'`" ]

then

echo "kernel does not appear to support policy routing" >/dev/stderr

exit 1

}

# parse the command line, skipping stuff we don't know, until there's nothing left

while :

for arg in "$@"

case $arg in

-t) no_change="1"; shift 1;;

*) shift 1 ;;

esac

continue 2

done

break

done

CheckDeps

echo ""

[ -z "$no_change" ] && \

cat /dev/null > /etc/iproute2/rt_tables

for rt in "${!rt_tables[@]}"

echo "adding routing table: $rt ${rt_tables[$rt]}"

[ -z "$no_change" ] && \

echo "$rt ${rt_tables[$rt]}" >> /etc/iproute2/rt_tables

done

interfaces="`ip addr show | sed -n 's/^[0-9][0-9]*: $[^:]*$: .*$/\1/p'`"

# reset the command-line arguments with the interface names

set -- $interfaces

ShowTables "BEFORE"

echo "EXEC RULES AND ROUTES"

{

for dev in "$@"

for ip_mask in `ip addr show $dev | sed -n 's/^\s*inet $[0-9\.\/]*$ .*$/\1/p'`

ip=${ip_mask%/*}

mask=${ip_mask#*/}

[ -z "$ip" -o -z "$mask" ] && echo "error reading \"ip addr show $dev\" info" >/dev/stderr

[ "$mask" -eq "32" ] && net="$ip" || \

net="`ipcalc -n $ip/$mask|sed -n 's/^network[=:][[:space:]]*$[0-9\.]*$.*$/\1/ip'`"

[ -z "$net" ] && echo "error reading \"ipcalc -n $ip/$mask\" output" >/dev/stderr

RouteNet $dev $ip $net/$mask `NetInfo $net/$mask`

done

# remove possible duplicate default routes etc from multiple ips on same network

# sort reverse to exec the deletes first, then the adds

} | sort -u -r | while read line

echo "$line"

[ -z "$no_change" ] && \

eval $line >/dev/null 2>&1

done

ShowTables "AFTER"

ip route flush cache

echo "FORWARDING"

sysctl -a 2>/dev/null | grep '^net\.ipv4\..*\.forwarding'

echo ""