How to Set Up SSH Keys on Ubuntu

SSH, or secure shell, is an encrypted protocol used to administer and communicate with servers. When working with an Ubuntu server, chances are you will spend most of your time in a terminal session connected to your server through SSH.

In this guide, we’ll focus on setting up SSH keys for an Ubuntu installation. SSH keys provide a secure way of logging into your server and are recommended for all users. We’ll cover everything from generating your first key pair to handling multiple keys for different services and users, alongside essential troubleshooting tips to keep you connected.

The first step is to create a key pair on the client machine (usually your computer):

ssh-keygen

By default recent versions of ssh-keygen will create a 3072-bit RSA key pair, which is secure enough for most use cases (you may optionally pass in the -b 4096 flag to create a larger 4096-bit key).

ssh-keygen -t ed25519

ssh-keygen -t rsa -b 3072

After entering the command, you should see the following output:

Output
Generating public/private rsa key pair.
Enter file in which to save the key (/your_home/.ssh/id_rsa):

Press enter to save the key pair into the .ssh/ subdirectory in your home directory, or specify an alternate path.

If you had previously generated an SSH key pair, you may see the following prompt:

Output
/home/your_home/.ssh/id_rsa already exists.
Overwrite (y/n)?

If you choose to overwrite the key on disk, you will not be able to authenticate using the previous key anymore. Be very careful when selecting yes, as this is a destructive process that cannot be reversed.

You should then see the following prompt:

Output
Enter passphrase (empty for no passphrase):

Here you optionally may enter a secure passphrase, which is highly recommended. A passphrase adds an additional layer of security to prevent unauthorized users from logging in. To learn more about security, consult our tutorial on How To Configure SSH Key-Based Authentication on a Linux Server.

You should then see the output similar to the following:

Output
Your identification has been saved in /your_home/.ssh/id_rsa
Your public key has been saved in /your_home/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:/hk7MJ5n5aiqdfTVUZr+2Qt+qCiS7BIm5Iv0dxrc3ks user@host
The key's randomart image is:
+---[RSA 3072]----+
|                .|
|               + |
|              +  |
| .           o . |
|o       S   . o  |
| + o. .oo. ..  .o|
|o = oooooEo+ ...o|
|.. o *o+=.*+o....|
|    =+=ooB=o.... |
+----[SHA256]-----+

You now have a public and private key that you can use to authenticate. The next step is to place the public key on your server so that you can use SSH-key-based authentication to log in.

The quickest way to copy your public key to the Ubuntu host is to use a utility called ssh-copy-id. Due to its simplicity, this method is highly recommended if available. If you do not have ssh-copy-id available to you on your client machine, you may use one of the two alternate methods provided in this section (copying via password-based SSH, or manually copying the key).

Copying the Public Key Using ssh-copy-id

The ssh-copy-id tool is included by default in many operating systems, so you may have it available on your local system. For this method to work, you must already have password-based SSH access to your server.

To use the utility, you specify the remote host that you would like to connect to, and the user account that you have password-based SSH access to. This is the account to which your public SSH key will be copied.

The syntax is:

ssh-copy-id username@remote_host

You may see the following message:

Output
The authenticity of host '203.0.113.1 (203.0.113.1)' can't be established.
ECDSA key fingerprint is fd:fd:d4:f9:77:fe:73:84:e1:55:00:ad:d6:6d:22:fe.
Are you sure you want to continue connecting (yes/no)? yes

This means that your local computer does not recognize the remote host. This will happen the first time you connect to a new host. Type “yes” and press ENTER to continue.

Next, the utility will scan your local account for the id_rsa.pub key that we created earlier. When it finds the key, it will prompt you for the password of the remote user’s account:

Output
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
username@203.0.113.1's password:

Type in the password (your typing will not be displayed, for security purposes) and press ENTER. The utility will connect to the account on the remote host using the password you provided. It will then copy the contents of your ~/.ssh/id_rsa.pub key into a file in the remote account’s home ~/.ssh directory called authorized_keys.

You should see the following output:

Output
Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'username@203.0.113.1'"
and check to make sure that only the key(s) you wanted were added.

At this point, your id_rsa.pub key has been uploaded to the remote account. You can continue on to Step 3.

Copying the Public Key Using SSH

If you do not have ssh-copy-id available, but you have password-based SSH access to an account on your server, you can upload your keys using a conventional SSH method.

We can do this by using the cat command to read the contents of the public SSH key on our local computer and piping that through an SSH connection to the remote server.

On the other side, we can make sure that the ~/.ssh directory exists and has the correct permissions under the account we’re using.

We can then output the content we piped over into a file called authorized_keys within this directory. We’ll use the >> redirect symbol to append the content instead of overwriting it. This will let us add keys without destroying previously added keys.

The full command looks like this:

cat ~/.ssh/id_rsa.pub | ssh username@remote_host "mkdir -p ~/.ssh && touch ~/.ssh/authorized_keys && chmod -R go= ~/.ssh && cat >> ~/.ssh/authorized_keys"

You may see the following message:

Output
The authenticity of host '203.0.113.1 (203.0.113.1)' can't be established.
ECDSA key fingerprint is fd:fd:d4:f9:77:fe:73:84:e1:55:00:ad:d6:6d:22:fe.
Are you sure you want to continue connecting (yes/no)? yes

This means that your local computer does not recognize the remote host. This will happen the first time you connect to a new host. Type yes and press ENTER to continue.

Afterwards, you should be prompted to enter the remote user account password:

Output
username@203.0.113.1's password:

After entering your password, the content of your id_rsa.pub key will be copied to the end of the authorized_keys file of the remote user’s account. Continue on to Step 3 if this was successful.

Copying the Public Key Manually

If you do not have password-based SSH access to your server available, you will have to complete the above process manually.

We will manually append the content of your id_rsa.pub file to the ~/.ssh/authorized_keys file on your remote machine.

To display the content of your id_rsa.pub key, type this into your local computer:

cat ~/.ssh/id_rsa.pub

You will see the key’s content, which should look something like this:

Output
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqql6MzstZYh1TmWWv11q5O3pISj2ZFl9HgH1JLknLLx44+tXfJ7mIrKNxOOwxIxvcBF8PXSYvobFYEZjGIVCEAjrUzLiIxbyCoxVyle7Q+bqgZ8SeeM8wzytsY+dVGcBxF6N4JS+zVk5eMcV385gG3Y6ON3EG112n6d+SMXY0OEBIcO6x+PnUSGHrSgpBgX7Ks1r7xqFa7heJLLt2wWwkARptX7udSq05paBhcpB0pHtA1Rfz3K2B+ZVIpSDfki9UVKzT8JUmwW6NNzSgxUfQHGwnW7kj4jp4AT0VZk3ADw497M2G/12N0PPB5CnhHf7ovgy6nL1ikrygTKRFmNZISvAcywB9GVqNAVE+ZHDSCuURNsAInVzgYo9xgJDW8wUw2o8U77+xiFxgI5QSZX3Iq7YLMgeksaO4rBJEa54k8m5wEiEE1nUhLuJ0X/vh2xPff6SQ1BL/zkOhvJCACK6Vb15mDOeCSq54Cr7kvS46itMosi/uS66+PujOO+xt/2FWYepz6ZlN70bRly57Q06J+ZJoc9FfBCbCyYH7U/ASsmY095ywPsBo1XQ9PqhnN1/YOorJ068foQDNVpm146mUpILVxmq41Cj55YKHEazXGsdBIbXWhcrRf4G2fJLRcGUr9q8/lERo9oxRm5JFX6TCmj6kmiFqv+Ow9gI0x8GvaQ== demo@test

Access your remote host using whichever method you have available.

Once you have access to your account on the remote server, you should make sure the ~/.ssh directory exists. This command will create the directory if necessary, or do nothing if it already exists:

mkdir -p ~/.ssh

Now, you can create or modify the authorized_keys file within this directory. You can add the contents of your id_rsa.pub file to the end of the authorized_keys file, creating it if necessary, using this command:

echo public_key_string >> ~/.ssh/authorized_keys

In the above command, substitute the public_key_string with the output from the cat ~/.ssh/id_rsa.pub command that you executed on your local system. It should start with ssh-rsa AAAA....

Finally, we’ll ensure that the ~/.ssh directory and authorized_keys file have the appropriate permissions set:

chmod -R go= ~/.ssh

This recursively removes all “group” and “other” permissions for the ~/.ssh/ directory.

If you’re using the root account to set up keys for a user account, it’s also important that the ~/.ssh directory belongs to the user and not to root:

chown -R sammy:sammy ~/.ssh

In this tutorial our user is named sammy but you should substitute the appropriate username into the above command.

We can now attempt passwordless authentication with our Ubuntu server.

If you have successfully completed one of the procedures above, you should be able to log into the remote host without providing the remote account’s password.

The basic process is the same:

ssh username@remote_host

If this is your first time connecting to this host (if you used the last method above), you may see something like this:

Output
The authenticity of host '203.0.113.1 (203.0.113.1)' can't be established.
ECDSA key fingerprint is fd:fd:d4:f9:77:fe:73:84:e1:55:00:ad:d6:6d:22:fe.
Are you sure you want to continue connecting (yes/no)? yes

This means that your local computer does not recognize the remote host. Type “yes” and then press ENTER to continue.

In this prompt, the host key fingerprint is for the server you are connecting to (not the SSH key pair you generated on your local machine). Depending on how the server is configured, the host key type shown here may be ECDSA, Ed25519, or RSA.

If you did not supply a passphrase for your private key, you will be logged in immediately. If you supplied a passphrase for the private key when you created the key, you will be prompted to enter it now (note that your keystrokes will not display in the terminal session for security). After authenticating, a new shell session should open for you with the configured account on the Ubuntu server.

If key-based authentication was successful, continue on to learn how to further secure your system by disabling password authentication.

If you were able to log into your account using SSH without a password, you have successfully configured SSH-key-based authentication to your account. However, your password-based authentication mechanism is still active, meaning that your server is still exposed to brute-force attacks.

Before completing the steps in this section, make sure that you either have SSH-key-based authentication configured for the root account on this server, or preferably, that you have SSH-key-based authentication configured for a non-root account on this server with sudo privileges. This step will lock down password-based logins, so ensuring that you will still be able to get administrative access is crucial.

Once you’ve confirmed that your remote account has administrative privileges, log into your remote server with SSH keys, either as root or with an account with sudo privileges. Then, open up the SSH daemon’s configuration file:

sudo nano /etc/ssh/sshd_config

Inside the file, search for a directive called PasswordAuthentication. This line may be commented out with a # at the beginning of the line. Uncomment the line by removing the #, and set the value to no. This will disable your ability to log in via SSH using account passwords:

. . .
PasswordAuthentication no
. . .

Save and close the file when you are finished by pressing CTRL+X, then Y to confirm saving the file, and finally ENTER to exit nano. To actually activate these changes, we need to restart the sshd service:

sudo systemctl restart ssh

As a precaution, open up a new terminal window and test that the SSH service is functioning correctly before closing your current session:

ssh username@remote_host

Once you have verified your SSH service is functioning properly, you can safely close all current server sessions.

The SSH daemon on your Ubuntu server now only responds to SSH-key-based authentication. Password-based logins have been disabled.

Once you have confirmed that key-based authentication is working correctly, you should harden your server’s SSH configuration to enhance security. The two most important steps are disabling password authentication and disabling direct root login.

These changes are made in the SSH daemon configuration file, /etc/ssh/sshd_config. Open it with sudo privileges:

sudo nano /etc/ssh/sshd_config

Disabling Password Authentication

Disabling password authentication protects your server from brute-force attacks, where attackers try to guess user passwords.

Find the PasswordAuthentication directive. If it is commented out (starts with a #), uncomment it by removing the #. Set its value to no:

...
PasswordAuthentication no
...

This change ensures that only clients with a valid SSH key can authenticate.

Disabling Root Login

Allowing direct root login over SSH is a security risk because the root user is a known, high-value target. It’s a best practice to log in as a regular user and elevate privileges with sudo when needed.

Find the PermitRootLogin directive and set its value to no:

...
PermitRootLogin no
...

This prevents anyone from logging in directly as the root user over SSH.

Applying Changes

After making your changes, save the file and exit the editor (press CTRL+X, then Y, then ENTER in nano).

To apply the new configuration, you must restart the SSH service:

sudo systemctl restart ssh

To verify the changes without getting locked out, open a new terminal window and try to connect to the server. Your key-based login should work. You can also try connecting from a machine that does not have your SSH key, or try ssh root@remote_host, to confirm that password and root logins are correctly denied. Once you have verified the new settings, you can safely close your original session.

Beyond the standard setup, SSH offers powerful features for further hardening your security posture. These advanced techniques provide granular control over key usage, leverage hardware-based authentication, and enable secure access to remote network services.

Hardware Keys

For the highest level of security, you can use a physical hardware key, such as a YubiKey or other FIDO2/U2F-compliant device. This method stores your private key on a tamper-resistant hardware device, meaning the key material itself never touches your computer’s disk. Authentication requires not just possession of the key but also a physical action (touching the device), providing robust protection against key theft and malware.

1. Generate the Key: Ensure your hardware key is plugged into your machine. Use the ssh-keygen command with the type ed25519-sk or ecdsa-sk:

   ssh-keygen -t ed25519-sk
   
   

   This will prompt you to touch your hardware key to confirm the operation. The process generates two files: the public key (id_ed25519_sk.pub) and a private key handle (id_ed25519_sk). The private key handle is not the key itself but a pointer to the key stored securely on the hardware device.

2. Deploy the Public Key: Copy the contents of id_ed25519_sk.pub to your server’s ~/.ssh/authorized_keys file, just as you would with a standard key.

3. Authenticate: When you initiate an SSH connection using this key, your SSH client will prompt you to touch your hardware key to authorize the session. This physical interaction ensures that an attacker cannot use your key without physical access to your device.

Restricting Key Access

You can apply fine-grained restrictions to individual keys by adding options at the beginning of their corresponding lines in the ~/.ssh/authorized_keys file on the server. This is especially useful for automating tasks or granting limited access.

The format is: options public-key-string

Here are some of the most practical options:

Practical Example:

Imagine you need a key for an automated backup script running on a server at 198.51.100.5. You want this key to only be able to run the backup script and nothing else. Your entry in authorized_keys would look like this:

from="198.51.100.5",command="/opt/scripts/backup.sh",no-port-forwarding,no-agent-forwarding,no-X11-forwarding ssh-ed25519 AAAAC3... backup-script-key

This configuration ensures the key can only be used from the specified IP to run a single, predefined command, with all other SSH features disabled.

SSH Agent Forwarding

SSH agent forwarding is a mechanism that allows you to use your local SSH keys to authenticate to a second server from a machine you are already connected to, without copying your private key to the intermediate server.

• Use Case: The most common scenario is connecting through a “bastion” or “jump” host. You SSH from your local machine (A) to a bastion server (B), and from there, you need to connect to an internal server ©. Agent forwarding lets you use the key from machine A to authenticate to machine C.

• How to Use: To enable agent forwarding for a single connection, use the -A flag:

  ssh -A user@bastion-host
  
  

  Once logged into bastion-host, you can then SSH to the internal server, and your local agent will handle the authentication:

  ssh user@internal-server
  
  

• Security Implications: While convenient, agent forwarding carries a significant security risk. If the intermediate server (bastion-host) is compromised, an attacker with root privileges on that machine can hijack your agent’s connection socket. This would allow them to use your forwarded credentials to log in to any other server that your key has access to, for as long as your session is active. Only use agent forwarding when connecting to servers you fully trust. A more secure modern alternative is to use the ProxyJump directive in your ~/.ssh/config file.

SSH Tunneling (Port Forwarding)

SSH tunneling, or port forwarding, creates an encrypted channel to securely route network traffic between your local machine and a remote server. This is useful for accessing services that are not exposed to the public internet or for encrypting traffic for legacy applications.

Local Port Forwarding

Local port forwarding (-L) allows you to access a service on a remote network as if it were running on your local machine.

• Use Case: Accessing a database on a remote server that only allows connections from localhost (itself).

• Syntax: ssh -L local_port:destination_host:destination_port user@ssh_server

• Example: To forward port 3306 (MySQL) on remote_server to your local port 8888, run:

  ssh -L 8888:localhost:3306 user@remote_server
  
  

  Now, you can connect your local database client to localhost:8888, and the traffic will be securely tunneled to port 3306 on remote_server.

Remote Port Forwarding

Remote port forwarding (-R) does the opposite: it exposes a service running on your local machine to the remote server’s network.

• Use Case: Temporarily showing a web application running on your local development machine (localhost:3000) to a colleague who can access the remote server.

• Syntax: ssh -R remote_port:destination_host:destination_port user@ssh_server

• Example: To expose your local web server to port 9999 on remote_server, run:

  ssh -R 9999:localhost:3000 user@remote_server
  
  

Your colleague can now access http://remote_server:9999 in their browser, and the requests will be forwarded to your local machine on port 3000. Be cautious with this feature, as it can expose local services to a remote network.

When you work with SSH keys, you interact with several configuration and trust files on both the client and the server. Each file has a specific responsibility in the authentication and connection process. Understanding which file controls which behavior helps you troubleshoot authentication failures, avoid unintentionally using the wrong key, and make server-side changes with confidence.

At a high level:

• The SSH client (ssh) decides how to connect.
• The SSH server (sshd) decides whether to allow the connection.
• Key and trust files determine identity and host verification.

Knowing how these pieces fit together is essential for secure and predictable SSH behavior.

SSH client configuration (ssh)

SSH client settings (the machine you connect from) are controlled by OpenSSH client configuration files. These files influence how the ssh command behaves, which credentials it presents, and what defaults are applied.

Client configuration can be defined in:

• ~/.ssh/config: This per-user configuration file is the most commonly used location for customizing SSH behavior. It allows you to define host aliases, default usernames, ports, preferred authentication methods, and which private key file to use for a given destination. Settings here apply only to the current user and override system-wide defaults.

• /etc/ssh/ssh_config: This file defines system-wide client defaults for all users on the machine. It is typically managed by the operating system or administrators. Any options defined in a user’s ~/.ssh/config take precedence over values in this file.

On newer Ubuntu releases, you may also see configuration snippets in /etc/ssh/ssh_config.d/*.conf

These files are included by the main ssh_config using the Include directive. This modular structure allows packages or administrators to drop in additional configuration without modifying the primary file.

Configuration precedence (highest to lowest):

1. Command-line options (ssh -i, -p, etc.)
2. ~/.ssh/config
3. /etc/ssh/ssh_config.d/*.conf
4. /etc/ssh/ssh_config
5. Built-in OpenSSH defaults

This precedence order is important when troubleshooting unexpected client behavior.

The Managing Multiple SSH Keys section demonstrates how to use ~/.ssh/config with the IdentityFile directive to ensure the correct private key is used for each host.

If you are unsure which settings your SSH client will apply for a given host, you can print the fully resolved configuration (including defaults and included files) with:

ssh -G <remote_host> | less

This command is extremely useful for debugging because it shows the final values after all configuration files and overrides have been processed.

SSH server configuration (sshd)

SSH server settings (the machine you connect to) are controlled by the OpenSSH daemon configuration. These settings determine whether a connection attempt is accepted and which authentication methods are permitted.

Server configuration is primarily defined in:

• /etc/ssh/sshd_config: This is the main OpenSSH server configuration file. It controls critical security behavior such as allowed authentication methods, login restrictions, key usage, root login policy, and connection limits.

Common directives in this file include:

• PasswordAuthentication
• PubkeyAuthentication
• PermitRootLogin
• AllowUsers / AllowGroups
• Port

On newer Ubuntu releases, additional configuration may appear in /etc/ssh/sshd_config.d/*.conf

These snippet files are included by the main configuration via the Include directive and allow modular server configuration management.

In Step 4 of this tutorial, you modified the PasswordAuthentication directive in sshd_config to disable password-based logins and enforce key-based authentication.

sudo systemctl reload ssh

sudo systemctl restart ssh

Reloading is safer because it applies changes without dropping existing connections.

Key and trust files you will commonly see

Several files participate in SSH authentication and host verification. Understanding their roles helps diagnose most SSH issues.

• ~/.ssh/authorized_keys (on the server): This file contains the list of public keys allowed to authenticate as a specific user. Each line represents one trusted public key. When you use ssh-copy-id, your public key is appended to this file by default. File permissions are critical; it must typically be set to 600, and the .ssh directory should be 700, or sshd may ignore it.

• ~/.ssh/known_hosts (on the client): This file stores the public host keys (or their hashed fingerprints) of servers you have previously connected to. During connection, the SSH client compares the server’s presented host key against this file to detect potential man-in-the-middle attacks. If the host key changes unexpectedly, SSH will display a warning and refuse the connection by default.

• /etc/ssh/ssh_known_hosts (on the client, optional): This file provides a system-wide known hosts database for all users. It is commonly used in managed environments to pre-trust infrastructure hosts. Entries here are checked in addition to the user’s known_hosts.

Host keys vs user keys

It is very common to confuse a server’s host key with a user’s authentication key. They serve completely different purposes in the SSH trust model.

User keys (client identity)

User keys are the key pair you generate with ssh-keygen to authenticate as a specific user account.

Key characteristics:

• The private key remains on your local machine (for example, ~/.ssh/id_rsa or ~/.ssh/id_ed25519) and must be kept secret.
• The public key is copied to the server, typically into the target user’s ~/.ssh/authorized_keys.
• These keys prove who you are to the server.
• They are used during the authentication phase of the SSH handshake.

If the server accepts your public key and the client proves possession of the private key, login succeeds.

Host keys (server identity)

Host keys identify the SSH server itself and protect clients from connecting to impostor machines.

Key characteristics:

• Generated automatically when the SSH server is installed.

• Stored on the server under /etc/ssh/ssh_host_*.

• Public portions are typically named like:

  • /etc/ssh/ssh_host_ed25519_key.pub
  • /etc/ssh/ssh_host_rsa_key.pub

• Presented to the client during the initial connection handshake.

• Verified against the client’s known_hosts.

These keys prove which server you are connecting to.

When you connect to a server for the first time, SSH displays the host key fingerprint and asks you to verify it. After you accept it, the key is stored in known_hosts for future validation.

When managing an Ubuntu server, you might need to grant SSH access to multiple users, each with their own SSH key. This is a secure practice as it allows for individual revocation of access without affecting other users.

To set up SSH keys for another user on your Ubuntu server, assuming you have sudo privileges, follow these steps:

• Create a new user, if they don’t exist:

  sudo adduser <new_username>
  
  

  Follow the prompts to set a password and user information.

• Switch to the new user as you’ll perform the next steps as the new user to ensure correct ownership and permissions.

  sudo su - <new_username>
  
  

• Next, create the .ssh directory:

  mkdir -p ~/.ssh
  
  

• Set permissions for the .ssh directory. This is crucial for security.

  chmod 700 ~/.ssh
  
  

• Now we’ll add the new user’s public key. The new user needs to provide you with their public key string (e.g., from their id_rsa.pub file). You can then add it to their authorized_keys file. Replace <public_key_string_from_new_user> with the actual public key.

  echo "<public_key_string_from_new_user>" >> ~/.ssh/authorized_keys
  
  

• Set the permissions for authorized_keys:

  chmod 600 ~/.ssh/authorized_keys
  
  

• Finally, exit back to your admin user:

  exit
  
  

Now, the new user should be able to connect to the server using their private key and the username you created for them:

ssh <new_username>@your_server_ip

Each user’s public key should be stored in their respective ~/.ssh/authorized_keys file on the server.

It’s common to have multiple SSH keys for different purposes, such as one for your servers and another for Git hosting services (GitHub, GitLab, Bitbucket). The SSH agent can help you manage these without constantly specifying which key to use.

• First, generate a new key with a different name. When generating a new key, specify a different filename using the -f flag. For example, for a Git key:

  ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_git -C "your_email@example.com_git"
  
  

  This will create id_rsa_git (private) and id_rsa_git.pub (public).

• Next, add keys to the SSH agent. The SSH agent is a program that holds your private keys in memory, so you don’t have to enter your passphrase every time you use the key.

  • Start the SSH agent (if it’s not already running) using the command:

    eval "$(ssh-agent -s)"
    
    

  • Add your keys to the agent:

    ssh-add ~/.ssh/id_rsa
    ssh-add ~/.ssh/id_rsa_git
    
    

  • To check which keys are loaded in the agent, use:

    ssh-add -l
    
    

• Finally, configure SSH to use specific keys for different hosts. You can tell SSH which key to use for which host by creating or editing the ~/.ssh/config file on your local machine.

  nano ~/.ssh/config
  
  

  Add your entries like this:

  # Server access
  Host your_server_alias
      HostName your_server_ip_or_hostname
      User your_username_on_server
      IdentityFile ~/.ssh/id_rsa
  
  # Git access (e.g., GitHub)
  Host github.com
      HostName github.com
      User git # This is typically 'git' for GitHub
      IdentityFile ~/.ssh/id_rsa_git
  
  # Another Git service (e.g., GitLab)
  Host gitlab.com
      HostName gitlab.com
      User git
      IdentityFile ~/.ssh/id_rsa_gitlab
  

Now, when you connect to your_server_alias (e.g., ssh your_server_alias), SSH will automatically use ~/.ssh/id_rsa. Similarly, git@github.com connections will use ~/.ssh/id_rsa_git.

Losing a private SSH key can be a security concern and will immediately prevent you from logging into servers that rely on that key for authentication. Here’s what you should do:

• The most crucial security step is to revoke the old key:

  • If you have other means of accessing your server (e.g., another SSH key, password-based login, or direct console access), log in to the affected server(s) immediately.
  • Edit the ~/.ssh/authorized_keys file for the user whose private key was lost.
  • Find the line corresponding to the lost public key and delete it entirely. Save the file.
  • This prevents anyone who might find your lost private key from using it to access your server.

• On your local machine, generate a brand new SSH key pair:

  ssh-keygen -t rsa -b 3072
  
  

  Choose a secure passphrase.

• Copy the new public key to your server(s). Follow Step 2 - Copying the Public Key to Your Ubuntu Server to upload your newly generated public key (id_rsa.pub) to your server(s).

• If the lost key was used for services like Git hosting (GitHub, GitLab), you’ll need to update your SSH keys on those platforms as well. Go to the SSH key settings in your account on those services and replace the old public key with your new one.

• Reflect on how the key was lost. If your computer was compromised, consider a full security audit of your system. If the key was on a removable drive, ensure that drive is securely wiped or destroyed.

SSH connection errors can be frustrating, but they are often resolvable with a systematic approach. Here are some common issues and their solutions:

• Permissions Errors: Incorrect file permissions on your SSH keys or the ~/.ssh directory are a frequent cause of connection failures.

  • Ensure your private key (id_rsa or similar) has permissions set to 600 (read/write only for the owner):

    chmod 600 ~/.ssh/id_rsa
    
    

  • Ensure your public key (id_rsa.pub or similar) has restrictive permissions. Both 600 (read/write for owner only) and 644 (read/write for owner, read for others) are acceptable; if you previously ran chmod -R go= ~/.ssh, your public key will typically be 600, which is fine:

    chmod 600 ~/.ssh/id_rsa.pub
    
    

  • Ensure your ~/.ssh directory has permissions set to 700 (read/write/execute for owner):

    chmod 700 ~/.ssh
    
    

  • On the server, the ~/.ssh directory should also be 700, and authorized_keys should be 600. Also, ensure the ~/.ssh directory and its contents are owned by the user you are logging in as:

    sudo chown -R <username>:<username> ~/.ssh
    sudo chmod 700 ~/.ssh
    sudo chmod 600 ~/.ssh/authorized_keys
    
    

• Incorrect Username or Hostname: Double-check that you are using the correct username and IP address/hostname for your server.

  ssh username@your_server_ip
  
  

• SSH Daemon Not Running (Server-Side): The SSH service (sshd) might not be running on your server. Log in via another method (e.g., cloud console) and check its status:

  sudo systemctl status ssh
  
  

  If it’s not running, start it:

  sudo systemctl start ssh
  
  

• Firewall Issues: A firewall (either on your local machine, the server, or an intermediary network) might be blocking SSH connections (port 22 by default).

  • On Ubuntu, you can check the server’s firewall status with ufw:

    sudo ufw status
    
    

    If port 22 is not allowed, you can allow it:

    sudo ufw allow ssh
    sudo ufw enable # if not already enabled
    
    

  • If you’re connecting from a restricted network, ensure port 22 is open outbound.

• Verbose Output for Debugging: To get more detailed information about why a connection is failing, use the -v (verbose) flag with your SSH command. You can use -vv or -vvv for even more detail:

  ssh -v username@your_server_ip
  
  

  This output can often pinpoint the exact reason for the failure, such as authentication issues, permission problems, or network blocks.

1. Why use SSH keys instead of passwords?

SSH keys offer significantly stronger security compared to traditional passwords. They are much longer and more complex, making them virtually impossible to guess or brute-force. Additionally, SSH keys enable automated logins without requiring manual password entry, improving convenience and efficiency for scripts and continuous integration workflows.

2. Where is my SSH private key stored?

By default, your SSH private key is stored in the .ssh subdirectory within your user’s home directory on your local machine. The most common default filename is id_rsa (for RSA keys) or id_ed25519 (for Ed25519 keys). For example, on Linux, it would typically be located at ~/.ssh/id_rsa. It’s crucial that this file is protected with strict permissions (e.g., chmod 600) to prevent unauthorized access.

3. What if I lose my private key?

If you lose your private key, it’s essential to take immediate action to protect your servers. First, revoke the lost public key from the ~/.ssh/authorized_keys file on all servers where it was used. Then, generate a brand new SSH key pair on your local machine and copy the new public key to your servers. For a detailed guide on this process, refer to the What to Do if Your Private Key is Lost section above.

4. Can I use one key for multiple servers?

Yes, absolutely. It is a common and recommended practice to use a single SSH key pair to authenticate with multiple servers. You simply copy the public key part of your key pair to the ~/.ssh/authorized_keys file on each server you wish to access with that key. The private key remains securely on your local machine.

5. How do I add multiple keys?

You can manage multiple SSH keys on your local machine for different purposes (e.g., one for your servers, another for Git hosting services). This typically involves generating new keys with different filenames (e.g., ssh-keygen -f ~/.ssh/id_rsa_git) and then either adding them to your SSH agent or configuring your ~/.ssh/config file to specify which key to use for which host. For a comprehensive explanation, please see the Managing Multiple SSH Keys section in this guide.

By following this guide, you’ve successfully set up SSH-key based authentication on your Ubuntu server, enhancing your security and streamlining your login process. You’re now equipped to manage multiple SSH keys for various needs, onboard other users securely, and confidently troubleshoot common connection issues. This foundational knowledge in SSH key management will serve you well in maintaining a robust and secure server environment.

If you’d like to learn more about working with SSH, take a look at our SSH Essentials Guide.

Ready to explore more about SSH and further secure your server? Check out these DigitalOcean resources:

• How to Create an SSH Key in Linux: Easy Step-by-Step Guide
• SSH Essentials: Working with SSH Servers, Clients, and Keys
• Understanding the SSH Encryption and Connection Process