How to configure Ssl termination with HAProxy when using varnish + apache?

You set the X-Forwarded-For headers in HAProxy. If there is already set an X-Forwarded-For header other reverse proxies will always just add their own to it, the left-most or first address is the source address. You don’t have to think about that, anything that reads and uses X-Forwarded-For headers will sort that out automagically.

You also want to set the X-Forwarded-Proto so you can do all sorts of magic in Varnish, like redirecting traffic not using TLS without hitting your backend servers and separate the caches, as Varnish doesn’t talk TLS, which can lead to some interesting results, like images not being served up because they are requested over HTTP when the page is served over HTTPS.

Side question, are you using HAProxy to actually load balance between multiple backends? If not, why not just terminate the TLS connection in Apache, send that to Varnish and then back to Apache again?