HAproxy and DNS in the cloud

HAproxy is a great tool that we all know and love.(Well, in case you don’t…go here!).
It is, however, surprising how, even basic features, are not default.
In particular, today I stumbled upon the configuration for HAproxy for dynamic DNS resolution.
In most cloud environments, nodes are coming and going all the time, and this happens while we rely on DNS for things like nodes and service discovery. If we deploy HAproxy as a forwarder towards an address defined as FQDN (instead of IP address), the default behavior is somewhat unsatisfying. The software will cache the initial DNS resolution and will never attempt at resolving it. I understand the reasoning behind this but it very inconvenient.

Worst than that, if the FQDN defined in the configuration is not available at startup time, HAproxy will fail to start, returning an error: “could not resolve address.” This can be common if, for example, you’re deploying your whole infrastructure as Terraform files and include the proxy in them, where part of your system is ready but not all of it. On top of this, the documentation on the subject is a bit convoluted.

However, it is straightforward to configure your way out of it:. You need to stop checking the DNS at configuration time and to only do it at run-time ( >1.6 ). To achieve this, you need to edit the configuration file (haproxy.cfg) :
A) Defining the resolvers in the defaults:

resolvers dnsserver
  nameserver opendns  208.067.200.200:53
  hold valid 1s

B) Disabling the check at configuration time:

defaults
    default-server init-addr none

C) Setting the server in the backend section to use its own resolver and enabling the health-check for it:

  server site example.com:80 resolvers dnsserver check

At this point, Haproxy will behave as expected.
As usual, I hope this is useful to somebody 😉

References:

https://serverfault.com/questions/681000/force-haproxy-to-lookup-dns-for-backend-server
https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#init-addr
https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5