Anycast: The Interface

In previous parts of this series, I discussed the basics of packet networking and how IP routing can be hacked to provide a geographically redundant IP. In this installment, I’ll talk about one way to configure an anycast IP on a server, and why you want to do it that way.

Let’s say you want to create an anycast IP at 10.10.10.10. You want to have a server in two different cities that each “own” that IP. The best way to do this is to configure the Anycast IP on a loopback alias, instead of on a regular interface.

You do this on RHEL-based systems fairly trivially, by creating a new file:

$ cat /etc/sysconfig/network-scripts/ifcfg-lo:10
DEVICE=lo:10
ONBOOT=yes
IPADDR=10.10.10.10
NETMASK=255.255.255.255

So, once you’ve gotten that interface configured, just ensure your service is listening to it. One thing to keep in mind here is that we’re creating an alias to the typical lo loopback interface. This interface is the one that has the 127.0.0.1 address on it. This is interesting because it brings up another corollary with other things: there’s nothing special about any particular IP address. 192.168.*.*, and 127.0.0.1 are only special by convention. Similarly, there’s nothing special about any given Layer 3-capable interface, including the loopback device. You can just as easily add an alias to the loopback interface with a non-local IP (as we’ve done here).

A good analogy is ASCII encoding: there’s nothing inherently special about the number 65 that ties it to “A”, it’s just a standardized convention we agree use. Likewise, there’s nothing special about any given char that prevents it from holding any given number between -128 and 127.

At any rate, the next step is to add a static route to your router directed at that IP. On a Cisco router, this would look like this:

RouterA# ip route 10.10.10.10 255.255.255.255 10.20.20.20 name dns.example.org

…assuming your “real” network card (that is, eth0) has 10.20.20.20 as it’s IP address. That’s it, you should now be able to reach your service from the Anycast IP.

So why do you want to do it that way, rather than just give the 10.10.10.10 address directly to a secondary network interface (i.e. different card)? Turns out there are some operational reasons for this:

The router will keep it’s route to your Anycast IP as long as there’s an active interface on the right VLAN. What this means in practice is that it’s often impossible to remove the route to 10.10.10.10 without totally shutting down the network card (that is, not just run ifdown on the interface, but modprobe -r on the driver module). Depending on your router/switch topology, even that may not work to remove the route.
You don’t need to “waste” another port on your switch. A decent 48-port, 10/100/1000, managed, Cisco switch costs about $2,500 online. That means each port costs $50, so don’t use them if you don’t have to.

You may have noticed the problem with operational reasoning #1: namely, how do you remove a static route once you’ve configured it on your router?

Some ways to do that are covered in the next installment:

If you like this, pass it along:

Like this:

5 thoughts on “Anycast: The Interface”