In my experience we've ran into issues trying to use both an internal wordpress plugin with an outside cache like Varnish.

Overall you are better at selecting one and sticking with it and alot of that also depends on what other plugins and functionality you have installed on your wordpress.

I'd recommend using WordPress Super Cache:
http://wordpress.org/extend/plugins/wp-super-cache/

And then reviewing system performance and making adjustments if necessary. Also if you make both changes at once and start running into issues you'll have a harder time of figuring out which one is at fault.

Setting up Varnish and W3TC is relatively easy and can be done in 20-30 minutes, the only issue you may run into is with tokens that are generated by database management plugins or software.

These tokens are not supposed to be cached and unless you modify the VCL to pass, you won't be able to log-in and use them. You'll still be able to log-in to WordPress, but if you use the Adminer plugin (either the WP plugin or stand-alone), SQLBuddy, phpMyAdmin, or other software, you won't unless you add that pass to the VCL. I've still not managed to find the right line to add, though if you need a starting point, we use the configuration below on quite a few deployments.

https://github.com/nicolargo/varnish-nginx-wordpress

It works well and it's really copy & paste.

Or maybe we dont need W3TC at all?

W3TC gives you access to Varnish tools like clearing the cache, etc. I've had good experience with it.

W3TC is a caching framework that manages policies for your page / minify / object / database / CDN (and now fragment) caching implementations. There's also a connector that will send purge requests to Varnish, so it's helpful to run W3TC alongside Varnish, yes.

But more to the point, Varnish obviate the other caching layers that a serious WPO approach includes. I suppose you can cobble together a solution to do the above with different plugins, but I'm obviously biased against this approach since we (W3 EDGE) make W3TC.

I'm not sure if this thread will notify me of responses, but you're welcome to drop us a note if you have any questions: http://www.w3-edge.com/contact/

With my Nginx running Varnish, I installed WordPress Super Cache in my Wordpress and it prompted me to setup .Htaccess mod-rewrite when I enable
"Use mod_rewrite to serve cache files. (Recommended)" then I realize I'm not on apache.

Is it possible to add .htaccess file to my wordpress root with Nginx running?

@PigGy: No, nginx does not support htaccess files. See if this helps: https://github.com/pothi/WordPress-Nginx/blob/master/globals/wpsc.conf