In order to have Varnish pass on the real client IP to your Apache access log, you’ll need to edit your Varnish configuration (/etc/varnish/default.vcl on Ubuntu) to add an X-Forwarded-For header. Find the vcl_recv section and added the following:

sub vcl_recv {
  unset req.http.X-Forwarded-For;
  set req.http.X-Forwarded-For = client.ip;
}

(Note: If you are using Varnish < 4.0 change unset to remove as the syntax is different.)

Than, in your Apache Virtual Host, set a CustomLog format:

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{User-agent}i\"" varnishcombined
CustomLog ${APACHE_LOG_DIR}/access.log varnishcombined

Finally, restart both Apache and Varnish for the changes to take effect:

sudo service varnish restart
sudo service apache2 restart