Deny access to directory, allow access to files NGINX

I’m trying to put my directories behind a htpasswd file, but allow any user to access the files in said directory.

So far I have this

location ~ ^/files\/.*(?=\.)$  {
  auth_basic            "Restricted";
  auth_basic_user_file  /media/sdh1/home/qwertyspace/www/qwertyspace.zeus/public_html/files/.htpasswd;
  autoindex on;
}

#allow retrieval of any individual image via URL without auth
location ~* \..*$ {
  autoindex off;
}

Based off this: https://serverfault.com/questions/574348/how-can-i-password-protect-a-directory-with-nginx-but-not-the-files-within-the

But it still asks for a password when trying to download a file.