How to Find SSH Keys on Ubuntu

SSH keys provide a secure method for authenticating to remote servers without passwords. For Ubuntu users, managing these keys efficiently is essential for both security and convenience. This comprehensive guide will walk you through locating, viewing, and managing SSH keys on your Ubuntu system.

Table of Contents

What Are SSH Keys?

SSH keys function as cryptographic key pairs that enable secure authentication between systems. Unlike passwords, which can be compromised through brute force attacks, SSH keys use asymmetric encryption to provide significantly stronger security.

Key components of SSH authentication

SSH authentication relies on a public-private key pair system:

The private key remains securely stored on your local machine
The public key is placed on remote servers you wish to access
When you connect, the server verifies your identity using these keys

SSH keys come in several types, each with different security characteristics:

RSA keys: Most widely used, typically 3072-4096 bits in length
ECDSA keys: Offers comparable security with smaller key sizes
Ed25519 keys: A newer algorithm providing excellent security and performance

Using SSH keys offers several advantages over password authentication, including enhanced security, passwordless login convenience, and support for automated scripts.

Default SSH Key Locations in Ubuntu

In Ubuntu, SSH keys are stored in a specific location with a standardized directory structure.

The ~/.ssh directory

SSH keys are typically found in the ~/.ssh directory within your home folder. This is a hidden directory (indicated by the leading dot) containing several important files:

id_rsa: Your private RSA key
id_rsa.pub: Your public RSA key (shares this name format for other key types as well)
id_ed25519: Your private Ed25519 key (if created)
id_ed25519.pub: Your public Ed25519 key (if created)
known_hosts: Records host keys from servers you’ve connected to
authorized_keys: Contains public keys authorized to log into your system
config: Optional file for SSH connection configurations

To access this hidden directory through the terminal, use standard navigation commands with the complete path. Alternatively, use the file browser with Ctrl+H to display hidden files and folders.

Checking for Existing SSH Keys

Before creating new SSH keys, it’s important to verify if you already have existing keys on your Ubuntu system.

Using the terminal to find SSH keys

Open a terminal window (Ctrl+Alt+T)
Run the following command to list the contents of your SSH directory:

ls -al ~/.ssh/

Examine the output for files with names like id_rsa, id_rsa.pub, id_ed25519, or other key pairs

If the command returns “No such file or directory,” it means the ~/.ssh directory doesn’t exist yet, indicating you haven’t created SSH keys on this system.

Interpreting the command output

The ls -al command displays detailed information about each file, including:

File permissions (should be -rw------- for private keys)
File owner and group
File size
Last modification date and time
Filename

To verify that your keys are valid and usable, check their file permissions and format. You can determine the key type and encryption level with:

ssh-keygen -l -f ~/.ssh/id_rsa

This command displays the bit length, fingerprint, and algorithm type of the specified key.

Viewing SSH Public Keys

Public keys are designed to be shared and therefore safe to view. Here are several methods to access and examine your public SSH keys.

Using the cat command

The simplest way to view your public key is with the cat command:

cat ~/.ssh/id_rsa.pub

The output will display the entire public key, which typically looks like:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCl... username@hostname

This string contains:

The key type identifier (ssh-rsa)
The key data (a long string of characters)
A comment (usually username@hostname) that helps identify the key’s origin

Alternative methods for viewing public keys

If you’ve loaded your keys into the SSH agent, you can view all loaded public keys with:

ssh-add -L

To check the fingerprint of your public key (useful for verification):

ssh-keygen -l -f ~/.ssh/id_rsa.pub

For sharing purposes, you can copy your public key to the clipboard:

cat ~/.ssh/id_rsa.pub | xclip -selection clipboard

Note that you may need to install xclip first with sudo apt install xclip.

Viewing SSH Private Keys

Private SSH keys require special handling and should never be shared with anyone. Here’s how to verify your private key’s existence and properties without compromising security.

Security considerations for private keys

When working with private keys, focus on verifying their existence and properties rather than viewing their contents. Exposing private key data creates significant security risks.

Commands to check private key properties

To verify a private key exists:

ls -la ~/.ssh/id_rsa

To check the private key’s permissions (should be 600):

stat -c "%a %n" ~/.ssh/id_rsa

To view information about the key without displaying its contents:

ssh-keygen -l -f ~/.ssh/id_rsa

Warning signs of compromised private keys

Be alert for these indicators that might suggest a private key has been compromised:

Incorrect permissions (anything other than 600)
Unexpected modification dates
The ability for other users to read the file
Unknown private keys in your SSH directory

If you suspect your private key has been compromised, generate a new key pair immediately and remove the compromised key from all services where it was used.

Generating New SSH Keys (if not found)

If you don’t have SSH keys or need to create new ones, the process is straightforward.

Complete ssh-keygen command syntax

The basic command to generate a new SSH key pair is:

ssh-keygen -t rsa -b 4096

Breaking down the options:

-t rsa specifies the key type (RSA in this example)
-b 4096 sets the key bit size (4096 provides strong security)

Step-by-step key generation process

Open a terminal with Ctrl+Alt+T
Run the ssh-keygen command with your preferred options
When prompted for a file location, press Enter to use the default location or specify a custom path
Enter a strong passphrase when prompted (or press Enter twice for no passphrase)
The system will generate both private and public keys and display their locations

Choosing key types and bit strength

For most modern systems, these are recommended options:

RSA: Use -t rsa -b 4096 for maximum compatibility
Ed25519: Use -t ed25519 for better security with newer systems

Setting up passphrases

A passphrase adds an additional layer of security to your private key:

Creates two-factor authentication (something you have and something you know)
Protects your key if the file is accidentally exposed
Can be securely managed with ssh-agent for convenience

After generation, verify your new keys with:

ls -la ~/.ssh

You should see both your private key (no extension) and public key (.pub extension).

SSH Key File Permissions

Correct file permissions for SSH keys are critical for both security and functionality.

Importance of correct SSH key permissions

SSH is designed to reject keys with overly permissive file permissions as a security measure. If your permissions are incorrect, you’ll likely encounter connection errors even with valid keys.

Recommended permission settings

The following permission settings are required for SSH to function properly:

SSH directory: 700 (drwx——)
Private keys: 600 (-rw——-)
Public keys: 644 (-rw-r–r–)
authorized_keys file: 600 (-rw——-)
known_hosts file: 644 (-rw-r–r–)

Commands to fix incorrect permissions

To set these permissions correctly:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/authorized_keys
chmod 644 ~/.ssh/known_hosts

Verifying permission settings

After applying these commands, verify the permissions are correct:

ls -la ~/.ssh

If you receive errors like “permissions are too open” during SSH connections, incorrect permissions are likely the cause.

Using Your SSH Keys

Once you’ve located or generated your SSH keys, you can use them to authenticate with remote Ubuntu servers.

Adding your public key to remote servers

The easiest method is using ssh-copy-id:

ssh-copy-id username@remote_host

This automatically copies your public key to the remote server’s authorized_keys file.

If ssh-copy-id isn’t available, use this manual method instead:

cat ~/.ssh/id_rsa.pub | ssh username@remote_host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys"

After adding your key, test the connection:

ssh username@remote_host

You should connect without being prompted for a password.

Setting up the authorized_keys file correctly

On the remote server, ensure the authorized_keys file has the correct permissions:

chmod 600 ~/.ssh/authorized_keys

The directory structure should be:

~/.ssh/authorized_keys

With ownership set to your user account:

chown -R username:username ~/.ssh

This setup ensures that SSH key authentication will work properly while maintaining security.

Managing Multiple SSH Keys

As you work with multiple servers or services, you may need several different SSH keys.

Creating named key pairs

Generate specific keys for different services:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/github_key
ssh-keygen -t rsa -b 4096 -f ~/.ssh/work_server_key

Using the SSH config file

Create or edit ~/.ssh/config to specify which key to use for which host:

# Personal GitHub account
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_key
    
# Work server
Host work-server
    HostName work.example.com
    User admin
    IdentityFile ~/.ssh/work_server_key
    Port 2222

With this configuration, you can simply use ssh work-server and the correct key will be used automatically.

Using ssh-agent to manage multiple keys

The SSH agent can hold multiple keys in memory:

eval $(ssh-agent -s)
ssh-add ~/.ssh/id_rsa
ssh-add ~/.ssh/github_key

To verify loaded keys:

ssh-add -l

This approach allows you to use multiple keys without entering passphrases repeatedly.

Troubleshooting SSH Key Issues

Even with proper setup, SSH key authentication can sometimes encounter problems.

Common “permission denied” errors

If you receive “Permission denied (publickey)” errors:

Check key permissions:
```
ls -la ~/.ssh
```

Fix with:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub

Verify the public key is in the remote server’s authorized_keys file:
```
ssh username@remote_host "cat ~/.ssh/authorized_keys"
```

Debugging SSH connection problems

For detailed debugging information, use verbose mode:

ssh -vvv username@remote_host

This output shows each step of the authentication process, making it easier to identify where things go wrong.

Key format compatibility issues

If you’re having trouble with older servers:

Generate RSA keys instead of newer formats:
```
ssh-keygen -t rsa -b 4096
```
Check server logs for more information:
- Ubuntu: /var/log/auth.log
- RHEL/CentOS: /var/log/secure

Server configuration issues

If the server doesn’t accept key authentication:

Check if PubkeyAuthentication is enabled in /etc/ssh/sshd_config on the server
Ensure the server’s SSH service is running:
```
sudo systemctl status sshd
```
Verify the server is configured to allow public key authentication:
```
grep PubkeyAuthentication /etc/ssh/sshd_config
```
It should show:
```
PubkeyAuthentication yes
```

If necessary, edit the configuration and restart the SSH service:

sudo systemctl restart sshd

These steps address most common SSH key issues on Ubuntu systems.

Advanced SSH Key Management

For more sophisticated use cases, consider these advanced techniques.

Using ssh-agent for convenient authentication

Start ssh-agent automatically in your ~/.bashrc:

if [ -z "$SSH_AUTH_SOCK" ] ; then
    eval $(ssh-agent -s)
    ssh-add ~/.ssh/id_rsa >/dev/null 2>&1
fi

This allows you to enter your passphrase once per session rather than each time you connect.

Implementing key rotation policies

Security best practices recommend regular key rotation:

Generate new keys periodically (every 6-12 months)
Update the authorized_keys file on all servers
Remove old keys after successful testing of new keys

Backing up SSH keys securely

Create an encrypted backup of your .ssh directory:

tar -czf ssh-backup.tar.gz ~/.ssh
gpg -c ssh-backup.tar.gz

Store the encrypted backup securely and delete the unencrypted version.

Using hardware security keys

For maximum security, consider using hardware security keys like YubiKey that can store SSH keys directly on the device. This prevents key extraction even if your system is compromised.

Security Best Practices for SSH Keys

Implement these practices to keep your SSH keys and systems secure.

Key length and algorithm recommendations

Use RSA keys with at least 3072 bits (4096 recommended)
Consider Ed25519 keys for better security and performance
Avoid legacy DSA keys when possible

Passphrase policies

Always use strong passphrases for SSH keys in production environments
Use different passphrases for different keys
Consider a password manager to store complex passphrases

Private key protection strategies

Never share private keys between users or systems
Don’t email or send private keys over unsecured channels
Set appropriate file permissions (600)
Consider encrypting your home directory for additional protection

Key revocation procedures

If a key is compromised:

Remove it from all authorized_keys files immediately
Generate new replacement keys
Audit server logs for unauthorized access

Monitoring SSH key usage

Enable verbose SSH server logging
Use tools like fail2ban to detect and block brute force attempts
Regularly audit authorized_keys files across your infrastructure

VPS Manage Service Offer

If you don’t have time to do all of this stuff, or if this is not your area of expertise, we offer a service to do “VPS Manage Service Offer”, starting from $10 (Paypal payment). Please contact us to get the best deal!