UFW not allowing specified ports

I have this

Status: active
Logging: on (low)
Default: allow (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere
80/tcp (Apache)            ALLOW IN    Anywhere
80                         ALLOW IN    Anywhere
443                        ALLOW IN    Anywhere
22/tcp (v6)                ALLOW IN    Anywhere (v6)
80/tcp (Apache (v6))       ALLOW IN    Anywhere (v6)
80 (v6)                    ALLOW IN    Anywhere (v6)
443 (v6)                   ALLOW IN    Anywhere (v6)

If I do ufw insert 1 deny from IP it does not work, the IP is still allowed, I assumed it is because default incoming is allow! But should it though? Should not this rue override the default rule?

And whenever I run ufw default deny incoming, which is the default configuration, I cannot access my server anymore, regardless of all the custom rules I added.

I ran ufw reset and also iptables -F, and did the following:

ufw allow apache ufw allow ssh

And I could not connect unless I changed ufw default incoming to allow

Note: I think, maybe this is because I ran iptables -F, I had to because I added some custom rules to iptables directly, not through ufw, and I wanted to start over

Please advise.