Well it was really me, but I have no idea what happened... Here's the TL;DR
I did everything properly according to https://www.digitalocean.com/community/articles/how-to-set-up-a-postfix-e-mail-server-with-dovecot
However this broke a lot. anytime I type my web URL it redirects to /mail and if I try x.com/index.html it automatically goes to /mail, which isn't there and I cannot seem to find anything wrong in the apache2 config.d. I also tried adding .htaccess files and the same error still.
If I try to access a page that uses cgi-bin I get an authentication required popup, it says it requires authentication.
I just want a nice simple solution for email, and this really blew up.
These answers are provided by our Community. If you find them useful, show some love by clicking the heart.
If you run into issues leave a comment, or add your own answer to help others.