Building a billion user load balancer

At Facebook, we make intensive use of Python in our infrastructure. Back in mid-2012, our engineers started to work on replacing the hardware load balancers that were powering Facebook with an in-house brew that would leverage Linux IPVS. Python was naturally chosen as the language to control, configure and manage the load balancers. By mid-2013, the whole of Facebook layer 4 load balancers in our datacenters had been replaced with Shiv and have been handling traffic ever since.

At the core of Shiv, we use libraries open-sourced by Facebook like Sparts [0], to set up a Thrift service, run and manage tasks, and gnlpy [1], to control IPVS via netlink. Add some python-fu to the mix and you can get a layer 4 load balancer that serves traffic for 1.65 billion people.

In this poster I will explain how our L4 load balancers are architected and how we use Python to leverage IPVS, setup individual VIPs and manage their backends, and make those load balancers operate in a fully automated way.

[0] https://github.com/facebook/sparts

[1] https://github.com/facebook/gnlpy

[2] https://www.youtube.com/watch?v=MKgJeqF1DHw