Installed Certbot, but now only homepage is accessible via HTTPS while all other pages access over HTTPS are broken

I installed certbot software on my DigitalOcean droplet, only the homepage is accessible via HTTPS, access to other pages via https:// links across the server fails, with a consistent error:

Not Found

The requested URL was not found on this server. Apache/2.4.41 (Ubuntu) Server at mysite.com Port 443

I followed the guide in this link: https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-20-04

mysite.com.conf

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName mysite.com
        ServerAlias www.mysite.com
        DocumentRoot /var/www/mysite.com/mysite-web-app/public

        <Directory /var/www/mysite.com/mysite-web-app/public>
            Options Indexes FollowSymLinks
            AllowOverride All
            Require all granted
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        <IfModule mod_dir.c>
            DirectoryIndex index.php index.pl index.cgi index.html index.xhtml index.htm
        </IfModule>

RewriteEngine on
RewriteCond %{SERVER_NAME} =www.mysite.com [OR]
RewriteCond %{SERVER_NAME} =mysite.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

mysite.com-le-ssl.conf

<IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerAdmin webmaster@localhost
    ServerName mysite.com
    ServerAlias www.mysite.com
    DocumentRoot /var/www/mysite.com/mysite-web-app/public
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/mysite.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mysite.com/privkey.pem
</VirtualHost>
</IfModule>