How to Configure SSH Key Authentication on Linux

Have you ever gotten tired of typing your password every time you connect to your server?

I sure did. After setting up several Linux servers for my websites and projects, I wanted a better way to log in — one that was faster and more secure. That’s when I discovered SSH key authentication. It made my life much easier.

In this article, I’ll show you how to set it up step by step. Whether you’re hosting a website or just learning Linux, this guide will help.

What is SSH?

Let’s start with the basics.

SSH stands for Secure Shell. It’s a tool that lets you connect to a remote computer or server using a command line. You can think of it as a secure tunnel between your computer and another computer, even if that one is far away.

Most Linux users use SSH to log into servers, especially if they manage websites or apps.

Usually, SSH asks for a password. But typing that password every time can get annoying. Worse, if someone figures out your password, they can break in.

This is where SSH keys come in.

What are SSH Keys?

SSH keys are a safer way to log in.

Instead of using a password, you use a pair of keys:

• A private key, which stays on your computer.
• A public key, which you copy to the server.

These two keys work together. The server checks if your private key matches the public one. If it does, you’re allowed in — no password needed.

It’s kind of like a lock and key. Only your private key can unlock your account.

Here’s why I use SSH keys:

• They are more secure than passwords.
• They are faster — no need to type anything.
• I can log into several servers without remembering different passwords.

Let’s Get Started: What You Need

You’ll need:

• A Linux or Mac computer (or Windows with WSL or Git Bash)
• A remote Linux server with SSH access
• Access to the terminal

Step 1: Check If You Already Have an SSH Key

Before creating a new key, check if you already have one.

Open your terminal and run:

ls ~/.ssh

If you see files like id_rsa and id_rsa.pub, you already have a key pair. You can use them or create a new one.

Step 2: Create a New SSH Key Pair

If you want a fresh key, it’s easy to make one.

In the terminal, type:

ssh-keygen

It will ask:

Enter file in which to save the key (/home/yourname/.ssh/id_rsa):

Press Enter to accept the default.

Then it asks for a passphrase. This is like a password for your key. You can leave it empty or add one for extra protection.

After that, you’ll see:

Your identification has been saved...
Your public key has been saved...

Now you have:

• A private key at ~/.ssh/id_rsa
• A public key at ~/.ssh/id_rsa.pub

Never share your private key.

Step 3: Copy the Public Key to Your Server

Now you need to tell the server to trust your key.

If your username on the server is user and the server IP is 192.168.1.100, use:

ssh-copy-id [email protected]

It may ask for your password just this one time. After that, you’re done.

If ssh-copy-id isn’t installed, you can do it manually:

cat ~/.ssh/id_rsa.pub

Copy the output, then log into your server and:

nano ~/.ssh/authorized_keys

Paste the key and save.

Step 4: Test the Connection

Now try logging in:

ssh [email protected]

If everything worked, you should be in — without a password.

I remember the first time it worked for me. I thought, “That’s it?” It felt like magic.

Step 5: Make It More Secure (Optional)

If you really want to lock down your server, you can turn off password login.

On your server, open the SSH config file:

sudo nano /etc/ssh/sshd_config

Find these lines and change them:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

Save the file, then restart SSH:

sudo systemctl restart sshd

Be careful — don’t do this unless you know your SSH key works. Otherwise, you could lock yourself out.

Why SSH Keys Are Better Than Passwords

Let’s compare:

When I work on five or six servers, SSH keys save me time every single day. I don’t worry about forgetting passwords or someone guessing them.

Tips for Managing SSH Keys

Here are a few things I learned along the way:

• Back up your private key. If you lose it, you can’t log in.
• Use a key manager if you have many keys.
• Keep your private key safe — don’t email it or store it online.
• Use different keys for different servers if needed.

You can also add your private key to the SSH agent:

ssh-add ~/.ssh/id_rsa

This saves it in memory, so you don’t need to type your passphrase again.

Common Problems (and How to Fix Them)

Things don’t always go smoothly. Here are two issues I ran into:

• Wrong permissions on .ssh folder or files. Run:

  chmod 700 ~/.ssh
  chmod 600 ~/.ssh/authorized_keys
  

• Wrong username or IP. Double-check them. Use whoami on the server to see your username.

If your key still doesn’t work, add -v to your SSH command for more info:

ssh -v [email protected]

Summary: What You Did

You just learned how to:

• Create SSH keys on your computer
• Copy the public key to a Linux server
• Log in without a password
• Improve security by disabling password login

Final Thoughts

SSH key authentication might seem hard at first, but once you do it once or twice, it becomes second nature.

When I was new to Linux, I avoided it because it looked too technical. But now, I can’t imagine running a server without it. It’s simple, fast, and safer than using passwords.

What about you? Have you tried using SSH keys yet?

Give it a shot and let me know how it goes.