What I now know about HAproxied Django DB connections, and wish I'd known sooner

Fri September 04, 01:55 PM–02:20 PM • Back to program
Start time 13:55
End time 14:20
Countdown link Open timer

Asynchronous Celery tasks that manipulate a MySQL/Galera database from a Django application can produce very interesting behavior when HAProxy is involved. This talk discusses these issues, and proposes remedies available to both application developers and infrastructure operators.

When you’re running a Django application, the following things are all pretty commonplace:

Now suppose your application doesn’t talk to your Galera cluster directly, but via HAProxy. That’s not exactly unheard of; in fact it’s an officially documented HA option for Galera. And, to complicate things further, this may be a feature of your setup that you, the application developer, don't necessarily have control over.

In such a scenario, you may be dealing with very "interesting" features of HAproxy which — if you are unfamiliar with them — can throw you very nasty curveballs.

Having been in the position where, together with my team, I was indeed unfamiliar with some of HAProxy's/Galera's intricacies and we were thus on he receiving end of those curveballs, I am taking this opportunity to share our findings so that our pain is someone else's gain. You have multiple options at your disposal — some in your infrastructure setup, and some in your Django application code. This talk covers both those angles.

Florian Haas

I run the Education business unit at City Network, and help people learn to use, understand, and deploy OpenStack. Together with my Education team I run academy.citycloud.com, a learning platform based on the open source Open edX software stack. I've worked exclusively with open source software since about 2002. I ran hastexo, a small professional services company specializing in OpenStack, Ceph, and Open edX, until our acquisition by City Network in October 2017. I have presented at several OpenStack/OpenInfra Summits, LinuxCnns, linux.conf.au's, OSCONs, and numerous other conferences and meetups