A pretty common approach is to set an expires time on all static assets:

location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
    expires 24hr;
}

If you want to just set it for the favicon, you can uses something like:

location = /favicon.ico {
    expires 24hr;
}

For more info, check out the documentation on the Nginx headers module.